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Preface 


This volume in the Texts in Logic and Games series was conceived as a 
ramification of the seventh conference on Logic and the Foundations of the 
Theory of Games and Decisions (LOFT7), which took place in Liverpool, 
in July 2006.1 

The LOFT conferences have been a regular biannual event since 1994. 
The first conference was hosted by the Centre International de Recherches 
Mathematiques in Marseille (France), the next four took place at the Inter- 
national Centre for Economic Research in Torino (Italy), the sixth confer- 
ence was hosted by the Graduate School of Management in Leipzig (Ger- 
many) and the most recent one took place at the University of Liverpool 
(United Kingdom).? 

The LOFT conferences are interdisciplinary events that bring together 
researchers from a variety of fields: computer science, economics, game the- 
ory, linguistics, logic, multi-agent systems, psychology, philosophy, social 
choice and statistics. In its original conception, LOFT had as its central 
theme the application of logic, in particular modal epistemic logic, to foun- 
dational issues in the theory of games and individual decision-making. Epis- 
temic considerations have been central to game theory for a long time. The 


1 The conference was organized by the editors of this volume with the assistance of a 
program committee consisting of Thomas Agotnes, Johan van Benthem, Adam Bran- 
denburger, Hans van Ditmarsch, Jelle Gerbrandy, Wojtek Jamroga, Hannes Leitgeb, 
Benedikt Lowe, Marc Pauly, Andrés Perea, Gabriella Pigozzi, Wlodek Rabinowicz, 
Hans Rott, and Krister Segerberg. 

Collections of papers from previous LOFT conferences can be found in a special issue of 
Theory and Decision (Vol. 37, 1994, edited by M. Bacharach and P. Mongin), the vol- 
ume Epistemic logic and the theory of games and decisions (edited by M. Bacharach, 
L.-A. Gérard-Varet, P. Mongin and H. Shin and published by Kluwer Academic, 1997), 
two special issues of Mathematical Social Sciences (Vols. 36 and 38, 1998, edited by 
G. Bonanno, M. Kaneko and P. Mongin), two special issues of Bulletin of Economic 
Research (Vol. 53, 2001 and Vol. 54, 2002, edited by G. Bonanno and W. van der 
Hoek), a special issue of Research in Economics, (Vol. 57, 2003, edited by G. Bo- 
nanno and W. van der Hoek), a special issue of Knowledge, Rationality and Action 
(part of Synthese, Vol. 147, 2005, edited by G. Bonanno) and the volume Proceedings 
of the 7th Conference on Logic and the Foundations of Game and Decision Theory 
(edited by G. Bonanno, W. van der Hoek and M. Wooldridge, University of Liverpool, 
2006). 
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expression “interactive epistemology” has been used in the game-theory lit- 
erature to refer to the analysis of decision making by agents involved in a 
strategic interaction, when these agents recognize each other’s intelligence 
and rationality. What is relatively new is the realization that the tools and 
methodologies that were used in game theory are closely related to those al- 
ready used in other fields, notably computer science and philosophy. Modal 
logic turned out to be the common language that made it possible to bring 
together different professional communities. It became apparent that the 
insights gained and the methodologies employed in one field could benefit 
researchers in other fields. Indeed, new and active areas of research have 
sprung from the interdisciplinary exposure provided by the LOFT confer- 
ences.’ 

Over time the scope of the LOFT conference has broadened to encom- 
pass a wider range of topics, while maintaining its focus on the general 
issue of rationality and agency. Topics that have fallen within the LOFT 
umbrella include epistemic and temporal logic, theories of information pro- 
cessing and belief revision, models of bounded rationality, non-monotonic 
reasoning, theories of learning and evolution, mental models, etc. 

The papers collected in this volume reflect the general interests and 
interdisciplinary scope of the LOFT conferences. 


The paper by Alexandru Baltag and Sonja Smets falls within the recent 
literature that deals with belief revision and update within the Dynamic 
Epistemic Logic paradigm. The authors develop a notion of doxastic action 
general enough to cover many examples of multi-agent communication ac- 
tions encountered in the literature, but also flexible enough to deal with both 
static and dynamic belief revision. They discuss several epistemic notions: 
knowledge, belief and conditional belief. For the latter they distinguish 
between the statement ‘if informed that P, the agent would believe that 
Q was the case (before the learning)’ and the statement ‘if informed that 
P, the agent would come to believe that Q is the case (in the world after 
the learning)’. They also study a “safe belief’ operator meant to express a 
weak notion of “defeasible knowledge”: it is belief that is persistent under 
revision with any true information. Baltag and Smets provide a complete 
axiomatization of the logic of conditional belief, knowledge and safe belief. 
In the second part of the paper the authors discuss dynamic belief revision 
in the context of action models. 


The paper by Giacomo Bonanno deals with the question of what choices 
are compatible with rationality of the players and common belief of ratio- 
nality. He takes a syntactic approach and defines rationality axiomatically. 


3 There is substantial overlap between the LOFT community and the community of 
researchers who are active in another regular, biannual event, namely the conferences 
on Theoretical Aspects of Rationality and Knowledge (TARK). 
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Furthermore, he does not assume von Neumann-Morgenstern payoffs but 
merely ordinal payoffs, thus aiming for a more general theory of rationality 
in games. The author considers two axioms. The first says that a player is 
irrational if she chooses a particular strategy while believing that another 
strategy of hers is better. He shows that common belief of this weak notion 
of rationality characterizes the iterated deletion of pure strategies that are 
strictly dominated by another pure strategy. The second axiom says that a 
player is irrational if she chooses a particular strategy while believing that 
a different strategy is at least as good and she considers it possible that 
this alternative strategy is actually better than the chosen one. The author 
shows that common knowledge of this stronger notion of rationality char- 
acterizes the iterated deletion procedure introduced by Stalnaker (1994), 
restricted—once again—to pure strategies. 


The paper by Hans van Ditmarsch and Barteld Kooi investigates a dy- 
namic logic describing “epistemic events” that may change both the agents’ 
information (or beliefs) and what the authors call “the ontic facts” of the 
world (that is, objective, non-epistemic statements about the world). A 
sound and complete axiomatization is provided. Some original and inter- 
esting semantic results are also proved, in particular the fact that any model 
change can be simulated by “epistemic events”, and thus any consistent goal 
can be achieved by performing some such event. The authors illustrate their 
results in several examples, including card games and logical puzzles. 


The paper by Wiebe van der Hoek, Mark Roberts and Michael Wool- 
dridge extends the authors’ previous work on Alternating-time Temporal 
Logic and its ramifications. They extend it by introducing the notion of 
a legally possible strategy, that they oppose to a physically possible strat- 
egy, and define social belief as truth in all states that are (1) possible for 
the agent, and (2) are obtained from the initial state by a legally possible 
strategy. They use this framework to reason about social laws. In a system 
with social laws, every agent is supposed to refrain from performing certain 
forbidden actions. Rather than assuming that all agents abide by the law, 
the authors consider what happens if certain agents act socially, while oth- 
ers do not. In particular, they focus on the agents’ strategic abilities under 
such mixed conditions. 


The paper by Alexander Nittka and Richard Booth deals with the tra- 
ditional “static” belief revision setting, but with a different twist: rather 
than answering the question of how an agent should rationally change his 
beliefs in the light of new information, they address the question of what 
one can say about an agent who is observed in a belief change process. 
That is, the authors study the problem of how to make inferences about 
an agent’s beliefs based on observation of how that agent responded to a 
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sequence of revision inputs over time. They start by reviewing some earlier 
results for the case where the observation is complete in the sense that (1) 
the logical content of all formulas appearing in the observation is known, 
and (2) all revision inputs received by the agent during the observed period 
are recorded in the observation. They then provide new results for the more 
general case where information in the observation might be distorted due to 
noise or because some revision inputs are missing altogether. Their analysis 
is based on the assumption that the agent employs a specific, but plausible, 
belief revision framework when incorporating new information. 


The paper by R. Ramanujam and Sunil Simon deals with the most im- 
portant notion of non-cooperative game, namely extensive game. Extensive 
games provide a richer description of interactive situations than strategic- 
form games in that they make the order of moves and the information avail- 
able to a player when it is his turn to move explicit. A strategy for a player 
in an extensive game associates with every information set of that player a 
choice at that information set. The authors observe that the game position 
(or information set) may be only partially known, in terms of properties 
that the player can test for. Thus—they argue—strategies can be thought 
of as programs, built up systematically from atomic decisions like if b then 
a where b is a condition checked by the player to hold (at some game po- 
sition) and a is a move available to the player at that position. This leads 
them to propose a logical structure for strategies, where one can reason 
with assertions of the form “(partial) strategy ø ensures the (intermediate) 
condition a”. They present an axiomatization for the logic and prove its 
completeness. 


The paper by Giacomo Sillari contributes to the very recent and fast 
growing literature on the notion of (un)awareness. An open problem in 
this literature has been how to model the state of mind of an individual 
who realizes that he may be unaware of something, that is, the problem of 
formalizing the notion of “awareness of unawareness”. Sillari offers a solu- 
tion to this problem using a new system of first-order epistemic logic with 
awareness. He also offers a philosophical analysis of awareness structures 
and proves that a certain fragment of the first-order epistemic language with 
awareness operators is decidable. 


The papers went through a thorough refereeing and editorial process. 
The editors would like to thank the many referees who provided invaluable 
help and the authors for their cooperation during the revision stage. 


Davis, CA & Liverpool G.B. W.v.d.H. M.W. 


A Qualitative Theory of Dynamic Interactive 
Belief Revision 


Alexandru Baltag! 
Sonja Smets?’ 


1 Computing Laboratory 
Oxford University 
Oxford OX1 3QD, United Kingdom 


2 Center for Logic and Philosophy of Science 
Vrije Universiteit Brussel 
Brussels, B1050, Belgium 


3 IEG Research Group on the Philosophy of Information 
Oxford University 
Oxford OX1 3QD, United Kingdom 


baltag@comlab.ox.ac.uk, sonsmets@vub.ac.be 


Abstract 


We present a logical setting that incorporates a belief-revision mecha- 
nism within Dynamic-Epistemic logic. As the “static” basis for belief 
revision, we use epistemic plausibility models, together with a modal 
language based on two epistemic operators: a “knowledge” modality 
K (the standard S5, fully introspective, notion), and a “safe belief” 
modality O (“weak”, non-negatively-introspective, notion, capturing 
a version of Lehrer’s “indefeasible knowledge”). To deal with “dy- 
namic” belief revision, we introduce action plausibility models, repre- 
senting various types of “doxastic events”. Action models “act” on 
state models via a modified update product operation: the “Action- 
Priority” Update. This is the natural dynamic generalization of AGM 
revision, giving priority to the incoming information (i.e., to “ac- 
tions”) over prior beliefs. We completely axiomatize this logic, and 
show how our update mechanism can “simulate”, in a uniform man- 
ner, many different belief-revision policies. 


1 Introduction 


This paper contributes to the recent and on-going work in the logical com- 
munity [2, 14, 24, 8, 10, 9, 7] on dealing with mechanisms for belief re- 
vision and update within the Dynamic-Epistemic Logic (DEL) paradigm. 
DEL originates in the work of Gerbrandy and Groeneveld [30, 29], an- 
ticipated by Plaza in [44], and further developed by numerous authors 


Giacomo Bonanno, Wiebe van der Hoek, Michael Wooldridge (eds.). Logic and the Founda- 
tions of Game and Decision Theory (LOFT 7). Texts in Logic and Games 3, Amsterdam 
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(6, 31, 22, 4, 23, 39, 5, 15, 16] etc. In its standard incarnation, as pre- 
sented e.g., in the recent textbook [25], the DEL approach is particularly 
well fit to deal with complex multi-agent learning actions by which groups of 
interactive agents update their beliefs (including higher-level beliefs about 
the others’ beliefs), as long as the newly received information is consistent 
with the agents’ prior beliefs. On the other hand, the classical AGM theory 
and its more recent extensions have been very successful in dealing with the 
problem of revising one-agent, first-level (factual) beliefs when they are con- 
tradicted by new information. So it is natural to look for a way to combine 
these approaches. 

We develop here a notion of dozastic actions', general enough to cover 
most examples of multi-agent communication actions encountered in the 
literature, but also flexible enough to deal with (both static and dynamic) 
belief revision, and in particular to implement various “belief-revision poli- 
cies” in a unified setting. Our approach can be seen as a natural extension 
of the work in [5, 6] on “epistemic actions”, incorporating ideas from the 
AGM theory along the lines pioneered in [2] and [24], but using a qualitative 
approach based on conditional beliefs, in the line of [50, 20, 19, 14]. 

Our paper assumes the general distinction, made in [24, 8, 14], between 
“dynamic” and “static” belief revision. It is usually acknowledged that 
the classical AGM theory in [1, 28] (and embodied in our setting by the 
conditional belief operators BPQ) is indeed “static”, in the sense that it 
captures the agent’s changing beliefs about an unchanging world. But in 
fact, when we take into account all the higher-level beliefs, the “world” 
(that these higher-level beliefs are about) includes all agent’s (real) beliefs.” 
Thus, such a world is always changed by our changes of beliefs! So we can 
better understand a belief conditional on P as capturing the agent’s beliefs 
after revising with P about the state of the world before the revision: the 
statement BPQ says that, if agent a would learn P, then she would come 
to believe that Q was the case (before the learning). In contrast, “dynamic” 
belief revision uses dynamic modalities to capture the agent’s revised beliefs 
about the world as it is after revision: |! P|BaQ says that after learning P, 
agent a would come to believe that Q is the case (in the world after the 
learning). The standard alternative [37] to the AGM theory calls this belief 
update, but like the AGM approach, it only deals with “first-level” (factual) 
beliefs from a non-modal perspective, neglecting any higher-order “beliefs 
about beliefs”. As a result, it completely misses the changes induced (in our 
own or the other agents’ epistemic-doxastic states) by the learning actions 
themselves (e.g., the learning of a Moore sentence, see Section 3). This 


1 Or “doxastic events”, in the terminology of [14]. 

2 To verify that a higher-level belief about another belief is “true” we need to check the 
content of that higher-level belief (i.e., the existence of the second, lower-level belief) 
against the “real world”. So the real world has to include the agent’s beliefs. 
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is reflected in the acceptance in [37] of the AGM “Success Axiom”: in 
dynamic notation, this is the axiom |! P] BaP (which cannot accommodate 
Moore sentences). Instead, the authors of [37] exclusively concentrate on 
the possible changes of (ontic) facts that may have occurred during our 
learning (but not due to our learning). In contrast, our approach to belief 
update (following the DEL tradition) may be thought of as “dual” to the 
one in [37]: we completely neglect here the ontic changes®, considering only 
the changes induced by “purely doxastic” actions (learning by observation, 
communication, etc.). 

Our formalism for “static” revision can best be understood as a modal- 
logic implementation of the well-known view of belief revision in terms of 
conditional reasoning [50, 52]. In [8] and [10], we introduced two equivalent 
semantic settings for conditional beliefs in a multi-agent epistemic context 
(conditional doxastic models and epistemic plausibility models), taking the 
first setting as the basic one. Here, we adopt the second setting, which is 
closer to the standard semantic structures used in the literature on modeling 
belief revision [34, 49, 52, 27, 19, 14, 17]. We use this setting to define 
notions of knowledge KaP, belief BaP and conditional belief BRP. Our 
concept of “knowledge” is the standard $5-notion, partition-based and fully 
introspective, that is commonly used in Computer Science and Economics, 
and is sometimes known as “Aumann knowledge”, as a reference to [3]. The 
conditional belief operator is a way to “internalize”, in a sense, the “static” 
(AGM) belief revision within a modal framework: saying that, at state s, 
agent a believes P conditional on Q is a way of saying that Q belongs to a’s 
revised “theory” (capturing her revised beliefs) after revision with P (of a’s 
current theory/beliefs) at state s. Our conditional formulation of “static” 
belief revision is close to the one in [50, 47, 19, 20, 45]. As in [19], the 
preference relation is assumed to be well-preordered; as a result, the logic 
CDL of conditional beliefs is equivalent to the strongest system in [19]. 

We also consider other modalities, capturing other “doxastic attitudes” 
than just knowledge and conditional belief. The most important such no- 
tion expresses a form of “weak (non-introspective) knowledge” O,P, first 
introduced by Stalnaker in his modal formalization [50, 52] of Lehrer’s de- 
feasibility analysis of knowledge [40, 41]. We call this notion safe belief, to 
distinguish it from our (Aumann-type) concept of knowledge. Safe belief 
can be understood as belief that is persistent under revision with any true 
information. We use this notion to give a new solution to the so-called 
“Paradox of the Perfect Believer”. We also solve the open problem posed 
n [19], by providing a complete axiomatization of the “static” logic KO of 
conditional belief, knowledge and safe belief. In a forthcoming paper, we 


3 But our approach can be easily modified to incorporate ontic changes, along the lines 
of [15]. 
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apply the concept of safe belief to Game Theory, improving on Aumann’s 
epistemic analysis of backwards induction in games of perfect information. 

Moving thus on to dynamic belief revision, the first thing to note is 
that (unlike the case of “static” revision), the doxastic features of the actual 
“triggering event” that induced the belief change are essential for under- 
standing this change (as a “dynamic revision”, i.e., in terms of the revised 
beliefs about the state of the world after revision). For instance, our beliefs 
about the current situation after hearing a public announcement (say, of 
some factual information, denoted by an atomic sentence p) are different 
from our beliefs after receiving a fully private announcement with the same 
content p. Indeed, in the public case, we come to believe that p is now 
common knowledge (or at least common belief). While, in the private case, 
we come to believe that the content of the announcement forms now our 
secret knowledge. So the agent’s beliefs about the learning actions in which 
she is currently engaged affect the way she updates her previous beliefs. 

This distinction is irrelevant for “static” revision, since e.g., in both cases 
above (public as well as private announcement) we learn the same thing 
about the situation that existed before the learning: our beliefs about that 
past situation will change in the same way in both cases. More generally, 
our beliefs about the “triggering action” are irrelevant, as far as our “static” 
revision is concerned. This explains a fact observed in [14], namely that 
by and large, the standard literature on belief revision (or belief update) 
does not usually make explicit the doxastic events that “trigger” the belief 
change (dealing instead only with types of abstract operations on beliefs, 
such as update, revision and contraction etc). The reason for this lies in the 
“static” character of AGM revision, as well as its restriction (shared with 
the “updates” of [37]) to one-agent, first-level, factual beliefs. 

A “truly dynamic” logic of belief revision has to be able to capture 
the doxastic-epistemic features (e.g., publicity, complete privacy etc.) of 
specific “learning events”. We need to be able to model the agents’ “dynamic 
beliefs”, i.e., their beliefs about the learning action itself: the appearance of 
this action (while it is happening) to each of the agents. In [5], it was 
argued that a natural way to do this is to use the same type of formalism 
that was used to model “static” beliefs: epistemic actions should be modeled 
in essentially the same way as epistemic states; and this common setting 
was taken there to be given by epistemic Kripke models. 

A similar move is made here in the context of our richer doxastic- 
plausibility structures, by introducing plausibility pre-orders on actions and 
developing a notion of “action plausibility models”, that extends the “epis- 
temic action models” from [5], along similar lines to (but without the quan- 
titative features of) the work in [2, 24]. 

Extending to (pre)ordered models the corresponding notion from [5], we 
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introduce an operation of product update of such models, based on the anti- 
lexicographic order on the product of the state model with the action model. 
The simplest and most natural way to define a connected pre-order on a 
Cartesian product from connected pre-orders on each of the components is 
to use either the lexicographic or the anti-lexicographic order. Our choice is 
the second, which we regard as the natural generalization of the AGM theory, 
giving priority to incoming information (i.e., to “actions” in our sense). This 
can also be thought of as a generalization of the so-called “mazimal-Spohn” 
revision. We call this type of update rule the “Action-Priority” Update. 
The intuition is that the beliefs encoded in the action model express the 
“ncoming” changes of belief, while the state model only captures that past 
beliefs. One could say that the new “beliefs about actions” are acting on 
the prior “beliefs about states”, producing the updated (posterior) beliefs. 
This is embedded in the Motto of Section 3.1: “beliefs about changes encode 
(and induce) changes of beliefs” . 

By abstracting away from the quantitative details of the plausibility 
maps when considering the associated dynamic logic, our approach to dy- 
namic belief revision is in the spirit of the one in [14]: instead of using 
“graded belief’ operators as in e.g., [2, 24], or probabilistic modal logic as 
in [39], both our account and the one in [14] concentrate on the simple, qual- 
itative language of conditional beliefs, knowledge and action modalities (to 
which we add here the safe belief operator). As a consequence, we obtain 
simple, elegant, general logical laws of dynamic belief revision, as natural 
generalizations of the ones in [14]. These “reduction laws” give a complete 
axtomatization of the logic of doxastic actions, “reducing” it to the “static” 
logic KO. Compared both to our older axiomatization in [10] and to the 
system in [2], one can easily see that the introduction of the safe belief 
operator leads to a major simplification of the reduction laws. 

Our qualitative logical setting (in this paper and in [8, 10, 9]), as well 
as van Benthem’s closely related setting in [14], are conceptually very dif- 
ferent from the more “quantitative” approaches to dynamic belief revision 
taken by Aucher, van Ditmarsch and Labuschagne [2, 24, 26], approaches 
based on “degrees of belief” given by ordinal plausibility functions. This is 
not just a matter of interpretation, but it makes a difference for the choice 
of dynamic revision operators. Indeed, the update mechanisms proposed 
in [49, 2, 24] are essentially quantitative, using various binary functions in 
transfinite ordinal arithmetic, in order to compute the degree of belief of the 
output-states in terms of the degrees of the input-states and the degrees of 
the actions. This leads to an increase in complexity, both in the computa- 
tion of updates and in the corresponding logical systems. Moreover, there 
seems to be no canonical choice for the arithmetical formula for updates, 
various authors proposing various formulas. No clear intuitive justification 
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is provided to any of these formulas, and we see no transparent reason to 
prefer one to the others. In contrast, classical (AGM) belief revision the- 
ory is a qualitative theory, based on natural, intuitive postulates, of great 
generality and simplicity. 

Our approach retains this qualitative flavor of the AGM theory, and 
aims to build a theory of “dynamic” belief revision of equal simplicity and 
naturality as the classical “static” account. Moreover (unlike the AGM 
theory), it aims to provide a “canonical” choice for a dynamic revision 
operator, given by our “Action Priority” update. This notion is a purely 
qualitative one*, based on a simple, natural relational definition. From a 
formal point of view, one might see our choice of the anti-lexicographic order 
as just one of the many possible options for developing a belief-revision- 
friendly notion of update. As already mentioned, it is a generalization 
of the “maximal-Spohn” revision, already explored in [24] and [2], among 
many other possible formulas for combining the “degrees of belief” of actions 
and states. But here we justify our option, arguing that our qualitative 
interpretation of the plausibility order makes this the only reasonable choice. 

It may seem that by making this choice, we have confined ourselves to 
only one of the bewildering multitude of “belief revision policies” proposed in 
the literature [49, 45, 48, 2, 24, 17, 14]. But, as argued below, this apparent 
limitation is not so limiting after all, but can instead be regarded as an 
advantage: the power of the “action model” approach is reflected in the 
fact that many different belief revision policies can be recovered as instances 
of the same type of update operation. In this sense, our approach can be 
seen as a change of perspective: the diversity of possible revision policies is 
replaced by the diversity of possible action models; the differences are now 
viewed as differences in input, rather than having different “programs”. For 
a computer scientist, this resembles “Currying” in lambda-calculus: if every 
“operation” is encoded as an input-term, then one operation (functional 
application) can simulate all operations.” In a sense, this is nothing but the 
idea of Turing’s universal machine, which underlies universal computation. 

The title of our paper is a paraphrase of Oliver Board’s “Dynamic In- 
teractive Epistemology” [19], itself a paraphrase of the title (“Interactive 
Epistemology”) of a famous paper by Aumann [3]. We interpret the word 
“interactive” as referring to the multiplicity of agents and the possibility 


4 One could argue that our plausibility pre-order relation is equivalent to a quantitative 
notion (of ordinal degrees of plausibility, such as [49]), but unlike in [2, 24] the way be- 
lief update is defined in our account does not make any use of the ordinal “arithmetic” 
of these degrees. 

5 Note that, as in untyped lambda-calculus, the input-term encoding the operation (i.e., 
our “action model”) and the “static” input-term to be operated upon (i.e., the “state 
model”) are essentially of the same type: epistemic plausibility models for the same 
language (and for the same set of agents). 
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of communication. Observe that “interactive” does not necessarily imply 
“dynamic”: indeed, Board and Stalnaker consider Aumann’s notion to be 
“static” (since it doesn’t accommodate any non-trivial belief revision). But 
even Board’s logic, as well as Stalnaker’s [52], are “static” in our sense: they 
cannot directly capture the effect of learning actions (but can only express 
“static” conditional beliefs). In contrast, our DEL-based approach has all 
the “dynamic” features and advantages of DEL: in addition to “simulat- 
ing” a range of individual belief-revision policies, it can deal with an even 
wider range of complex types of multi-agent learning and communication 
actions. We thus think it is realistic to expect that, within its own natural 
limits®, our Action-Priority Update Rule could play the role of a “universal 
machine” for qualitative dynamic interactive belief-revision. 


2 “Static” Belief Revision 


Using the terminology in [14, 8, 10, 9, 11], “static” belief revision is about 
pre-encoding potential belief revisions as conditional beliefs. A conditional 
belief statement BPQ can be thought of as expressing a “doxastic predis- 
position” or a “plan of doxastic action”: the agent is determined to believe 
that Q was the case, if he learnt that P was the case. The semantics for con- 
ditional beliefs is usually given in terms of plausibility models (or equivalent 
notions, e.g., “spheres”, “onions”, ordinal functions etc.) As we shall see, 
both (Aumann, $5-like) knowledge and simple (unconditional) belief can be 
defined in terms of conditional belief, which itself could be defined in terms 
of a unary belief-revision operator: *qP captures all the revised beliefs of 
agent a after revising (her current beliefs) with P. 

In addition, we introduce a safe belief operator 0,P, meant to express a 
weak notion of “defeasible knowledge” (obeying the laws of the modal logic 
54.3). This concept was defined in [52, 19] using a higher-order semantics 
(quantifying over conditional beliefs). But this is in fact equivalent to a 
first-order definition, as the Kripke modality for the (converse) plausibility 
relation. This observation greatly simplifies the task of completely axiom- 
atizing the logic of safe belief and conditional beliefs: indeed, our proof 
system KO below is a solution to the open problem posed in [19]. 


2.1 Plausibility models: The single agent case 


To warm up, we consider first the case of only one agent, a case which fits 
well with the standard models for belief revision. 

A single-agent plausibility frame is a structure (S,<), consisting of a 
set S of “states” and a “well-preorder” <, i.e., a reflexive, transitive binary 
6 E.g., our update cannot deal with “forgetful” agents, since “perfect recall” is in-built. 


But finding out what exactly are the “natural limits” of our approach is for now an 
open problem. 
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relation on S such that every non-empty subset has minimal elements. Using 
the notation 


Mince P := {s€ P:s < 3 for all s € P} 


for the set of <-minimal elements of P, the last condition says that: For 
every set P C S, if P # Ø then Mine # Ø. 

The usual reading of s < t is that “state s is at least as plausible as 
state t”. We keep this reading for now, though we will later get back to it 
and clarify its meaning. The “minimal states” in Min< P are thus the “most 
plausible states” satisfying proposition P. As usual, we write s < tiffs < t 
but t Ż s, for the “strict” plausibility relation (s is more plausible than t). 
Similarly, we write s S t iff both s < t and t < s, for the “equi-plausibility” 
(or indifference) relation (s and t are equally plausible). 


S-propositions and models. Given an epistemic plausibility frame S, 
an S-proposition is any subset P C S. Intuitively, we say that a state s 
satisfies the proposition P if s € P. Observe that a plausibility frame is just 
a special case of a relational frame (or Kripke frame). So, as it is standard 
for Kripke frames in general, we can define a plausibility model to be a 
structure S = (S,<,||-||), consisting of a plausibility frame (S, <) together 
with a valuation map ||-|| : ® — P(S), mapping every element of a given set 
® of “atomic sentences” into S-propositions. 


Interpretation. The elements of S will represent the possible states (or 
“possible worlds”) of a system. The atomic sentences p € ® represent 
“ontic” (non-doxastic) facts, that might hold or not in a given state. The 
valuation tells us which facts hold at which worlds. Finally, the plausibility 
relations < capture the agent’s (conditional) beliefs about the state of the 
system; if e.g., the agent was given the information that the state of the 
system is either s or t, she would believe that the system was in the most 
plausible of the two. So, if s < t, the agent would believe the real state was 
s; if t < s, she would believe it was t; otherwise (if s S t), the agent would 
be indifferent between the two alternatives: she will not be able to decide 
to believe any one alternative rather than the other. 


Propositional operators, Kripke modalities. For every model S, we 
have the usual Boolean operations with S-propositions 


PAQ:=PNQ, PVQ:=PUQ, 


=P :=S\P, P3>Q:=7-PVQ, 


as well as Boolean constants Ts := S and Ls := Ø. Obviously, one also 
introduces infinitary conjunctions and disjunctions. In addition, any binary 
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relation R C Sx S on S gives rise to a Kripke modality |R] : P(S) => P(S), 
defined by 
[R]Q := {s € S : Vt(sRt>t € Q)}. 


Accessibility relations for belief, conditional belief and knowledge. 
To talk about beliefs, we introduce a doxastic accessibility relation —, given 
by 

s — t iff t € Min< S. 


We read this as saying that: when the actual state is s, the agent believes 
that any of the states t with s — t may be the actual state. This matches 
the above interpretation of the preorder: the states believed to be possible 
are the minimal (i.e., “most plausible”) ones. 

In order to talk about conditional beliefs, we can similarly define a con- 
ditional doxastic accessibility relation for each S-proposition P C S: 


s—? tiff t€ Ming P. 


We read this as saying that: when the actual state is s, if the agent is given 
the information (that) P (is true at the actual state), then she believes that 
any of the states t with s — t may be the actual state. 

Finally, to talk about knowledge, we introduce a relation of epistemic 
possibility (or “indistinguishability” ) ~. Essentially, this is just the universal 
relation: 

s~tiff stes. 


So, in single-agent models, all the states in S are assumed to be “epistem- 
ically possible”: the only thing known with absolute certainty about the 
current state is that it belongs to S. This is natural, in the context of a 
single agent: the states known to be impossible are irrelevant from the point 
of doxastic-epistemic logic, so they can simply be excluded from our model 
S. (As seen below, this cannot be done in the case of multiple agents!) 


Knowledge and (conditional) belief. We define knowledge and (condi- 
tional) belief as the Kripke modalities for the epistemic and (conditional) 
doxastic accessibility relations: 


BeP := |>®?]P. 


We read KP as saying that the (implicit) agent knows P. This is “knowl- 
edge” in the strong Leibnizian sense of “truth in all possible worlds”. We 
similarly read BP as “P is believed” and B® P as “P is believed given (or 
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conditional on) Q”. As for conditional belief statements s € B® P, we inter- 
pret them in the following way: if the actual state is s, then after coming 
to believe that Q is the case (at this actual state), the agent will believe 
that P was the case (at the same actual state, before his change of belief). 
In other words, conditional beliefs B? give descriptions of the agent’s plan 
(or commitments) about what he will believe about the current state after 
receiving new (believable) information. To quote Johan van Benthem in 
[14], conditional beliefs are “static pre-encodings” of the agent’s potential 
belief changes in the face of new information. 


Discussion on interpretation. Observe that our interpretation of the 
plausibility relations is qualitative, in terms of conditional beliefs rather than 
“degrees of belief’: there is no scale of beliefs here, allowing for “interme- 
diary” stages between believing and not believing. Instead, all these beliefs 
are equally “firm” (though conditional): given the condition, something is 
either believed or not. To repeat, writing s < t is for us just a way to say 
that: if given the information that the state of the system is either s or t, 
the agent would believe it to be s. So plausibility relations are special cases 
of conditional belief. This interpretation is based on the following (easily 
verifiable) equivalence: 


s < tiff s € BHs} if t e BOM {s}. 


There is nothing quantitative here, no need for us to refer in any way 
to the “strength” of this agent’s belief: though she might have beliefs of 
unequal strengths, we are not interested here in modeling this quantitative 
aspect. Instead, we give the agent some information about a state of a 
virtual system (that it is either s or t) and we ask her a yes-or-no question 
(“Do you believe that virtual state to be s ?”); we write s < t iff the agent’s 
answer is “yes”. This is a firm answer, so it expresses a firm belief. “Firm” 
does not imply “un-revisable” though: if later we reveal to the agent that 
the state in question was in fact t, she should be able to accept this new 
information; after all, the agent should be introspective enough to realize 
that her belief, however firm, was just a belief. 

One possible objection against this qualitative interpretation is that our 
postulate that < is a well-preorder (and so in particular a connected pre- 
order) introduces a hidden “quantitative” feature; indeed, any such preorder 
can be equivalently described using a plausibility map as in e.g., [49], as- 
signing ordinals to states. Our answer is that, first, the specific ordinals 
will not play any role in our definition of a dynamic belief update; and 
second, all our postulates can be given a justification in purely qualitative 
terms, using conditional beliefs. The transitivity condition for < is just a 
consistency requirement imposed on a rational agent’s conditional beliefs. 
And the existence of minimal elements in any non-empty subset is simply 
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the natural extension of the above setting to general conditional beliefs, 
not only conditions involving two states: more specifically, for any possible 
condition P C S about a system S, the S-proposition Min< P is simply a 
way to encode everything that the agent would believe about the current 
state of the system, if she was given the information that the state satisfied 
condition P. 


Note on other models in the literature. Our models are the same as 
Board’s “belief revision structures” [19], i.e., nothing but “Spohn models” as 
in [49], but with a purely relational description. Spohn models are usually 
described in terms of a map assigning ordinals to states. But giving such a 
map is equivalent to introducing a well pre-order < on states, and it is easy 
to see that all the relevant information is captured by this order. 

Our conditions on the preorder < can also be seen as a semantic analogue 
of Grove’s conditions for the (relational version of) his models in [34]. The 
standard formulation of Grove models is in terms of a “system of spheres” 
(weakening Lewis’ similar notion), but it is equivalent (as proved in [34]) to 
a relational formulation. Grove’s postulates are still syntar-dependent, e.g., 
existence of minimal elements is required only for subsets that are defin- 
able in his language: this is the so-called “smoothness” condition, which is 
weaker than our “well-preordered” condition. We prefer a purely semantic 
condition, independent of the choice of a language, both for reasons of ele- 
gance and simplicity and because we want to be able to consider more than 
one language for the same structure.” So, following [19, 52] and others, we 
adopt the natural semantic analogue of Grove’s condition, simply requir- 
ing that every subset has minimal elements: this will allow our conditional 
operators to be well-defined on sentences of any extension of our logical 
language. 

Note that the minimality condition implies, by itself, that the relation < 
is both reflexive (i.e., s < s for all s € S) and connected? (i.e., either s < t or 
t < s, for all s,t € S). In fact, a “well-preorder” is the same as a connected, 
transitive, well-founded® relation, which is the setting proposed in [19] for 
a logic of conditional beliefs equivalent to our logic CDL below. Note also 
that, when the set S is finite, a well-preorder is nothing but a connected 
preorder. This shows that our notion of frame subsumes, not only Grove’s 
setting, but also some of the other settings proposed for conditionalization. 


T Imposing syntactic-dependent conditions in the very definition of a class of structures 
makes the definition meaningful only for one language; or else, the meaning of what, 
say, a plausibility model is won’t be robust: it will change whenever one wants to 
extend the logic, by adding a few more operators. This is very undesirable, since then 
one cannot compare the expressivity of different logics on the same class of models. 

8 In the Economics literature, connectedness is called “completeness”, see e.g., [19]. 

9 I.e., there exists no infinite descending chain so > s1 >---. 
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2.2 Multi-agent plausibility models 


In the multi-agent case, we cannot exclude from the model the states that 
are known to be impossible by some agent a: they may still be considered 
possible by a second agent b. Moreover, they might still be relevant for a’s 
beliefs/knowledge about what b believes or knows. So, in order to define 
an agent’s knowledge, we cannot simply quantify over all states, as we did 
above: instead, we need to consider, as usually done in the Kripke-model 
semantics of knowledge, only the “possible” states, i.e., the ones that are 
indistinguishable from the real state, as far as a given agent is concerned. It 
is thus natural, in the multi-agent context, to explicitly specify the agents’ 
epistemic indistinguishability relations ~a (labeled with the agents’ names) 
as part of the basic structure, in addition to the plausibility relations <,. 
Taking this natural step, we obtain epistemic plausibility frames (S, ~a, <q). 
As in the case of a single agent, specifying epistemic relations turns out to 
be superfluous: the relations ~a can be recovered from the relations <,. 
Hence, we will simplify the above structures, obtaining the equivalent setting 
of multi-agent plausibility frames (S, <a). 

Before going on to define these notions, observe that it doesn’t make 
sense anymore to require the plausibility relations <, to be connected (and 
even less sense to require them to be well-preordered): if two states s,¢ are 
distinguishable by an agent a, i.e., S £a t, then a will never consider both 
of them as epistemically possible in the same time. If she was given the 
information that the real state is either s or t, agent a will immediately know 
which of the two: if the real state was s, she would be able to distinguish this 
state from t, and would thus know the state was s; similarly, if the real state 
was t, she would know it to be t. Her beliefs will play no role in this, and it 
would be meaningless to ask her which of the two states is more plausible to 
her. So only the states in the same ~,-equivalence class could, and should, 
be <,-comparable; i.e., s Sa t implies s ~a t, and the restriction of <a to 
each ~,-equivalence class is connected. Extending the same argument to 
arbitrary conditional beliefs, we can see that the restriction of <, to each 
~q-equivalence class must be well-preordered. 


Epistemic plausibility frames. Let A be a finite set of labels, called 
agents. A epistemic plausibility frame over A (EPF, for short) is a struc- 
ture S = (S, ~a, <a)aeA, Consisting of a set S of “states”, endowed with a 
family of equivalence relations ~a, called epistemic indistinguishability re- 
lations, and a family of plausibility relations <,, both labeled by “agents” 
and assumed to satisfy two conditions: (1) <,-comparable states are ~a- 
indistinguishable (i.e., s <q t implies s ~a t); (2) the restriction of each 
plausibility relation <, to each ~,-equivalence class is a well-preorder. As 
before, we use the notation Min<, P for the set of <,-minimal elements 
of P. We write s <a t iff s <a t but t Ża s (the “strict” plausibility rela- 
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tion), and write s 4 t iff both s <a t and t <a s (the “equi-plausibility” 
relation). The notion of epistemic plausibility models (EPM, for short) is 
defined in the same way as the plausibility models in the previous section. 


Epistemic plausibility models. We define a (multi-agent) epistemic 
plausibility model (EPM, for short) as a multi-agent EPF together with 
a valuation over it (the same way that single-agent plausibility models were 
defined in the previous section). 

It is easy to see that our definition of EPFs includes superfluous infor- 
mation: in an EPF, the knowledge relation ~a can be recovered from the 
plausibility relation <,, via the following rule: 


S ~a t iff either s<, tort <q s. 


In other words, two states are indistinguishable for a iff they are comparable 
(with respect to <,). 

So, in fact, one could present epistemic plausibility frames simply as 
multi-agent plausibility frames. To give this alternative presentation, we use, 
for any preorder relation <, the notation ~ for the associated comparability 
relation 

~wi= sus 


(where > is the converse of <). A comparability class is a set of the form 
{t:s <tort < s}, for some state s. A relation < is called locally well- 
preordered if it is a preorder such that its restriction to each comparability 
class is well-preordered. Note that, when the underlying set S is finite, a 
locally well-preordered relation is nothing but a locally connected preorder: a 
preorder whose restrictions to any comparability class are connected. More 
generally, a locally well-preordered relation is the same as a locally connected 
and well-founded preorder. 


Multi-agent plausibility frames. A multi-agent plausibility frame 
(MPF, for short) is a structure (S, <a)aca, consisting of a set of states 
S together with a family of locally well-preordered relations <,, one for 
each agent a € A. Oliver Board [19] calls multi-agent plausibility frames 
“belief revision structures”. A multi-agent plausibility model (MPM, for 
short) is an MPF together with a valuation map. 


Bijective correspondence between EPFs and MPFs. Every MPF 
can be canonically mapped into an EPF, obtained by defining epistemic in- 
distinguishability via the above rule (~a := <a U 2a). Conversely, every 
EPF gives rise to an MPF, via the map that “forgets” the indistinguisha- 
bility structure. It is easy to see that these two maps are the inverse of 
each other. Consequently, from now on we identify MPFs and EPFs, and 
similarly identify MPMs and EPMs; e.g., we can talk about “knowledge”, 
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“(conditional) belief’ etc. in an MPM, defined in terms of the associated 
EPM. 

So from now on we identify the two classes of models, via the above 
canonical bijection, and talk about “plausibility models” in general. One 
can also see how this approach relates to another widely adopted definition 
for conditional beliefs; in [19, 24, 14], this definition involves the assumption 
of a “local plausibility” relation at a given state s <} t, to be read as: “at 
state w, agent a considers state s at least as plausible as state t ”. Given 
such a relation, the conditional belief operator is usually defined in terms 
that are equivalent to putting s >? t iff t € Mings P. One could easily 
restate our above definition in this form, by taking: 


s<? t iff either w%,t or s <qt. 


The converse problem is studied in [19], where it is shown that, if full intro- 
spection is assumed, then one can recover “uniform” plausibility relations 
<a from the relations <?’. 


Information cell. The equivalence relation ~, induces a partition of the 
state space S, called agent a’s information partition. We denote by s(a) the 
information cell of s in a’s partition, i.e., the ~,-equivalence class of s: 


sla) := {tE S: Ss ~a t}. 


The information cell s(a) captures all the knowledge possessed by the agent 
at state s: when the actual state of the system is s, then agent a knows 
only the state’s equivalence class s(a). 


Example 2.1. Alice and Bob play a game, in which an anonymous referee 
puts a coin on the table, lying face up but in such a way that the face is 
covered (so Alice and Bob cannot see it). Based on previous experience, (it 
is common knowledge that) Alice and Bob believe that the upper face is 
Heads (since e.g., they noticed that the referee had a strong preference for 
Heads). And in fact, they’re right: the coin lies Heads up. Neglecting the 
anonymous referee, the EPM for this example is the following model S: 


Here, the arrows represent converse plausibility relations > between distinct 
states only (going from less plausible to more plausible states): since these 
are always reflexive, we choose to skip all the loops for convenience. The 
squares represent the information cells for the two agents. Instead of labels, 
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we use dashed arrows and squares for Alice, while using continuous arrows 
and squares for Bob. In this picture, the actual state of the system is the 
state s on the left (in which H is true). Henceforth, in our other examples, 
we will refer to this particular plausibility model as S. 

By deleting the squares, we obtain a representation of the corresponding 
MPM, also denoted by S (where we now use labels for agents instead of 
different types of lines): 

He T 

e e 
Example 2.2. In front of Alice, the referee shows the face of the coin to 
Bob, but Alice cannot see the face. The EPM is now the following model W: 


l 

Z | 

kg ial | 
e ° 

| 

| 


while the MPM is 

He a T 

e e 
Since Bob now knows the state of the coin, his local plausibility relation 
consists only of loops, and hence we have no arrows for Bob in this dia- 
grammatic representation. 


(Conditional) doxastic appearance and (conditional) doxastic ac- 
cessibility. As in the previous section, we can define a doxastic and epis- 
temic accessibility relations, except that now we have to select, for each 
state s, the most plausible states in its information cell s(a) (instead of the 
most plausible in S). For this, it is convenient to introduce some notation 
and terminology: the doxastic appearance of state s to agent a is the set 


Sa := Ming, s(a) 


of the “most plausible” states that are consistent with the agent’s knowledge 
at state s. The doxastic appearance of s captures the way state s appears to 
the agent, or (in the language of Belief Revision) the agent’s current “theory” 
about the world s. We can extend this to capture conditional beliefs (in full 
generality), by associating to each S-proposition P C S and each state 
s € S the conditional doxastic appearance sẹ of state s to agent a, given 
(information) P. This can be defined as the S-proposition 
sẹ := Ming, s(a) N P 


a ` 
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given by the set of all <,-minimal states of s(a) N P: these are the “most 
plausible” states satisfying P that are consistent with the agent’s knowledge 
at state s. The conditional appearance SP gives the agent’s revised theory 
(after learning P) about the world s. We can put these in a relational form, 
by defining doxastic accessibility relations >a, >", as follows: 


s—,t iff tE Sa, 


so? tiffte st. 


Knowledge and (conditional) belief. As before, we define the knowl- 
edge and (conditional) belief operators for an agent a as the Kripke modal- 
ities for a’s epistemic and (conditional) doxastic accessibility relations: 


KaP := [~alP = {s € S: s(a) C P}, 
BaP := |[>a]P = {s€ S: sa CP}, 
BLP := |>$]P = {sE 9:88 C P}. 


We also need a notation for the dual of the K modality (“epistemic possi- 
bility” ): N 
KaP := 7AK.AP. 


Doxastic propositions. Until now, our notion of proposition is “local”, 
being specific to a given model: we only have “S-propositions” for each 
model S. As long as the model is fixed, this notion is enough for interpreting 
sentences over the given model. But, since later we will proceed to study 
systematic changes of models (when dealing with dynamic belief revision), 
we need a notion of proposition that is not confined to one model, but makes 
sense on all models: 

A dozastic proposition is a map P assigning to each plausibility model S 
some S-proposition Pg C S. We write s =s P, and say that the proposition 
P is true at s € S, iff s € (P)s. We skip the subscript and write s = P 
when the model is understood. 

We denote by Prop the family of all doxastic propositions. All the 
Boolean operations on S-propositions as sets can be lifted pointwise to op- 
erations on Prop: in particular, we have the “always true” T and “always 
false” L propositions, given by (L)s := @,(T)s := S, negation (4P)s := 
S \ Ps, conjunction (PAQ)s := PsN Qs, disjunction (PV Q)s := Ps UQs 
and all the other standard Boolean operators, including infinitary conjunc- 
tions and disjunctions. Similarly, we can define pointwise the epistemic and 
(conditional) doxastic modalities: (KaP)s := KaPs, (BaP)s := BaPs, 
(B2P)s := BÌSPs. It is easy to check that we have: BaP = B} P. Fi- 
nally, the relation of entailment P = Q between doxastic propositions is 
given pointwise by inclusion: P = Q iff Ps C Qs for all S. 
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2.3 Safe belief and the Defeasibility Theory of Knowledge 


Ever since Plato’s identification of knowledge with “true justified (or justi- 
fiable) belief” was shattered by Gettier’s celebrated counterexamples [32], 
philosophers have been looking for the “missing ingredient” in the Platonic 
equation. Various authors identify this missing ingredient as “robustness” 
(Hintikka [35]), “indefeasibility” (Klein [38], Lehrer [40], Lehrer and Paxson 
[41], Stalnaker [52]) or “stability” (Rott [46]). According to this defeasibility 
theory of knowledge (or “stability theory”, as formulated by Rott), a belief 
counts as “knowledge” if it is stable under belief revision with any new evi- 
dence: “if a person has knowledge, than that person’s justification must be 
sufficiently strong that it is not capable of being defeated by evidence that 
he does not possess” (Pappas and Swain [43]). 

One of the problems is interpreting what “evidence” means in this con- 
text. There are at least two natural interpretations, each giving us a concept 
of “knowledge”. The first, and the most common", interpretation is to take 
it as meaning “any true information”. The resulting notion of “knowledge” 
was formalized by Stalnaker in [52], and defined there as follows: “an agent 
knows that y if and only if ọ is true, she believes that y, and she continues 
to believe vy if any true information is received”. This concept differs from 
the usual notion of knowledge (“Aumann knowledge” ) in Computer Science 
and Economics, by the fact that it does not satisfy the laws of the modal 
system S5 (in fact, negative introspection fails); Stalnaker shows that the 
complete modal logic of this modality is the modal system $4.3. As we’ll 
see, this notion (“Stalnaker knowledge”) corresponds to what we call “safe 
belief?’ OP. On the other hand, another natural interpretation, considered 
by at least one author [46], takes “evidence” to mean “any proposition”, 
i.e., to include possible misinformation: “real knowledge” should be robust 
even in the face of false evidence. As shown below, this corresponds to our 
“knowledge” modality KP, which could be called “absolutely unrevisable 
belief’. This is a partition-based concept of knowledge, identifiable with 
“Aumann knowledge” and satisfying all the laws of S5. In other words, this 
last interpretation provides a perfectly decent “defeasibility” defense of S5 
and of negative introspection! 

In this paper, we adopt the pragmatic point of view of the formal lo- 
gician: instead of debating which of the two types of “knowledge” is the 
real one, we simply formalize both notions in a common setting, compare 
them, axiomatize the logic obtained by combining them and use their joint 
strength to express interesting properties. Indeed, as shown below, condi- 
tional beliefs can be defined in terms of knowledge only if we combine both 
the above-mentioned types of “knowledge”. 


10 This interpretation is the one virtually adopted by all the proponents of the defeasi- 
bility theory, from Lehrer to Stalnaker. 
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Knowledge as unrevisable belief. Observe that, for all propositions P, 
we have 


KaQ = ABQ 
P 


(where the conjunction ranges over all doxastic propositions), or equiva- 
lently, we have for every state s in every model S: 


s |H KaQ iff s| BPQ for all P. (2.1) 


This gives a characterization of knowledge as “absolute” belief, invariant 
under any belief revision: a given belief is “known” iff it cannot be revised, 
i.e., it would be still believed in any condition.!! Observe that this resem- 
bles the defeasibility analysis of knowledge, but only if we adopt the second 
interpretation mentioned above (taking “evidence” to include misinforma- 
tion). Thus, our “knowledge” is more robust than Stalnaker’s: it resists 
any belief revision, not capable of being defeated by any evidence (includ- 
ing false evidence). This is a very “strong” notion of knowledge (implying 
“absolute certainty” and full introspection), which seems to us to fit better 
with the standard usage of the term in Computer Science literature. Also, 
unlike the one in [52], our notion of knowledge is negatively introspective. 

Another identity!? that can be easily checked is: 


K,Q = B7QQ = B7Q L (2.2) 


(where L is the “always false” proposition). This captures in a different 
way the “absolute un-revisability” of knowledge: something is “known” if 
it is believed even if conditionalizing our belief with its negation. In other 
words, this simply expresses the impossibility of accepting its negation as 
evidence (since such a revision would lead to an inconsistent belief). 


Safe belief. To capture “Stalnaker knowledge”, we introduce the Kripke 
modality O, associated to the converse >, of the plausibility relation, going 
from any state s to all the states that are “at least as plausible” as s. For 
S-propositions P C S over any given model S, we put 


aP :=(>,|P={seES:te P for allt <a s}, 


and this induces pointwise an operator OP on doxastic propositions. We 
read s = O,P as saying that: at state s, agent a’s belief in P is safe; or at 


11 This of course assumes agents to be “rational” in a sense that excludes “fundamen- 
talist” or “dogmatic” beliefs, i.e., beliefs in unknown propositions but refusing any 
revision, even when contradicted by facts. But this “rationality” assumption is al- 
ready built in our plausibility models, which satisfy an epistemically friendly version 
of the standard AGM postulates of rational belief revision. See [8] for details. 

12 This identity corresponds to the definition of “necessity” in [50] in terms of doxastic 
conditionals. 
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state s, a safely believes that P. We will explain this reading below, but first 
observe that: Oa is an S4-modality (since >, is reflexive and transitive), but 
not necessarily $5; i.e., safe beliefs are truthful (OP |= P) and positively 
introspective (OP = O,0,P), but not necessarily negatively introspective: 
in general, -O,P aUaP. 


Relations between knowledge, safe belief and conditional belief. 
First, knowledge entails safe belief 


KaP [— aP d 


and safe belief entails belief 


aP — BaP, 


The last observation can be strengthened to characterize safe belief in a sim- 
ilar way to the above characterization (2.1) of knowledge (as belief invariant 
under any revision): safe beliefs are precisely the beliefs which are persistent 
under revision with any true information. Formally, this says that, for every 
state s in every model S, we have 


sH oQ iff. s| BEQ for every P such that s EP (2.3) 


We can thus see that safe belief coincides indeed with Stalnaker’s notion 
of “knowledge”, given by the first interpretation (“evidence as true informa- 
tion” ) of the defeasibility theory. As mentioned above, we prefer to keep the 
name “knowledge” for the strong notion (which gives absolute certainty), 
and call this weaker notion “safe belief’: indeed, these are beliefs that are 
“safe” to hold, in the sense that no future learning of truthful information 
will force us to revise them. 


Example 2.3 (Dangerous Knowledge). This starts with the situation in 
Example 2.1 (when none of the two agents has yet seen the face of the 
coin). Alice has to get out of the room for a minute, which creates an 
opportunity for Bob to quickly raise the cover in her absence and take a 
peek at the coin. He does that, and so he sees that the coin is Heads up. 
After Alice returns, she obviously doesn’t know whether or not Bob took a 
peek at the coin, but she believes he didn’t do it: taking a peek is against 
the rules of the game, and so she trusts Bob not to do that. The model is 
now rather complicated, so we ao represent the MPM: 


He T 
e 
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Let us call this model S’. The actual state s} is the one in the upper left 
corner, in which Bob took a peek and saw the coin Heads up, while the 
state t} in the upper right corner represents the other possibility, in which 
Bob saw the coin lying Tails up. The two lower states s and t represent 
the case in which Bob didn’t take a peek. Observe that the above drawing 
includes the (natural) assumption that Alice keeps her previous belief that 
the coin lies Heads up (since there is no reason for her to change her mind). 
Moreover, we assumed that she will keep this belief even if she’d be told 
that Bob took a peek: this is captured by the a-arrow from tį to s}. This 
seems natural: Bob’s taking a peek doesn’t change the upper face of the 
coin, so it shouldn’t affect Alice’s prior belief about the coin. 


In both Examples 2.1 and 2.3 above, Alice holds a true belief (at the 
real state) that the coin lies Heads up: the actual state satisfies BaH. In 
both cases, this true belief is not knowledge (since Alice doesn’t know the 
upper face), but nevertheless in Example 2.1, this belief is safe (although 
it is not known by the agent to be safe): no additional truthful information 
(about the real state s) can force her to revise this belief. (To see this, 
observe that any new truthful information would reveal to Alice the real 
state s, thus confirming her belief that Heads is up.) So in the model S 
from Example 2.1, we have s = O,H (where s is the actual state). In 
contrast, in Example 2.2, Alice’s belief (that the coin is Heads up), though 
true, is not safe. There is some piece of correct information (about the real 
state s1) which, if learned by Alice, would make her change this belief: we 
can represent this piece of correct information as the doxastic proposition 
H — K» H. It is easy to see that the actual state s{ of the model S’ 
satisfies the proposition BHS K»HT (since (H — KpH)s = {8}, t1, th} and 
the minimal state in the set s{(a)N {s4, 54, t4} = {s4, |, tg} is th, which 
satisfies T.) So, if given this information, Alice would come to wrongly 
believe that the coin is Tails up! This is an example of a dangerous truth: 
a true information whose learning can lead to wrong beliefs. 

Observe that an agent’s belief can be safe without him necessarily know- 
ing this (in the “strong” sense of knowledge given by K): “safety” (similarly 
to “truth”) is an external property of the agent’s beliefs, that can be as- 
certained only by comparing his belief-revision system with reality. Indeed, 
the only way for an agent to know a belief to be safe is to actually know it 
to be truthful, i.e., to have actual knowledge (not just a belief) of its truth. 
This is captured by the valid identity 


K,UaP = K.P. (2.4) 


In other words: knowing that something is safe to believe is the same as just 
knowing it to be true. In fact, all beliefs held by an agent “appear safe” to 
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him: in order to believe them, he has to believe that they are safe. This is 
expressed by the valid identity 


BUeP = BaP (2.5) 


saying that: believing that something is safe to believe is the same as just 
believing it. Contrast this with the situation concerning “knowledge”: in our 
logic (as in most standard doxastic-epistemic logics), we have the identity 


BaKaP = KaP. (2.6) 
So believing that something is known is the same as knowing it! 


The Puzzle of the Perfect Believer. The last identity is well-known 
and has been considered “paradoxical” by many authors. In fact, the so- 
called “Paradox of the Perfect Believer” in [33, 53, 36, 42, 54, 27] is based 
on it. For a “strong” notion of belief as the one we have here (“belief”? = 
belief with certainty), it seems reasonable to assume the following “axiom”: 


Bap > BaKay. (?) 
Putting this together with (2.6) above, we get a paradoxical conclusion: 
Bap > Kav. (?!) 


So this leads to a triviality result: knowledge and belief collapse to the same 
thing, and all beliefs are always true! One solution to the “paradox” is to 
reject (?), as an (intuitive but) wrong “axiom”. In contrast, various authors 
[53, 36, 27, 54] accept (?) and propose other solutions, e.g., giving up the 
principle of “negative introspection” for knowledge. 

Our solution to the paradox, as embodied in the contrasting identities 
(2.5) and (2.6), combines the advantages of both solutions above: the “ax- 
iom” (?) is correct if we interpret “knowledge” as safe belief Oa, since then 
(?) becomes equivalent to identity (2.5) above; but then negative introspec- 
tion fails for this interpretation! On the other hand, if we interpret “knowl- 
edge” as our K,-modality then negative introspection holds; but then the 
above “axiom” (?) fails, and on the contrary we have the identity (2.6). 

So, in our view, the paradox of the perfect believer arises from the confla- 
tion of two different notions of “knowledge”: “Aumann” (partition-based) 
knowledge and “Stalnaker” knowledge (i.e., safe belief). 


(Conditional) beliefs in terms of “knowledge” notions. An impor- 
tant observation is that one can characterize/define (conditional) beliefs 
only in terms of our two “knowledge” concepts (K and O): For simple 
beliefs, we have 


BaP = Ka aP = Oa aP, 
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recalling that K,P = 7K,—P is the Diamond modality for Ka, and O,.P = 
=0,—P is the Diamond for Og. 

The equivalence BaP = ©,U,P has recently been observed by Stalnaker 
in [52], who took it as the basis of a philosophical analysis of “belief” in terms 
of “defeasible knowledge” (i.e., safe belief). Unfortunately, this analysis does 
not apply to conditional belief: one can easily see that conditional belief 
cannot be defined in terms of safe belief only! However, one can generalize 
the identity B,P = K,O,P above, defining conditional belief in terms of 
both our “knowledge” concepts: 


BEQ = K.P > K,(PAO,(P > Q)). (2.7) 


2.4 Other modalities and doxastic attitudes 


From a modal logic perspective, it is natural to introduce the Kripke modal- 
ities [>a] and [©aa] for the other important relations (strict plausibility and 
equiplausibility): For S-propositions P C S over a given model S, we put 


[>a]P := {s€ S:te P for allt <, s}, 
[~,|P := {se S:teP forall t &, s}, 


and as before these pointwise induce corresponding operators on Prop. The 
intuitive meaning of these operators is not very clear, but they can be used to 
define other interesting modalities, capturing various “doxastic attitudes”. 


Weakly safe belief. We can define a weakly safe belief operator OW°**P 
in terms of the strict order by putting: 


weakp — PA[>,|P. 


Clearly, this gives us the following truth clause: 


s H Owe P iff st Pandt/ P forallt<s. 


But a more useful characterization is the following: 


s H OWeekQ iff: sK-ABP-=Q for every P such that s P. 


So “weakly safe beliefs” are beliefs which (might be lost but) are never re- 
versed (into believing the opposite) when revising with any true information. 


The unary revision operator. Using the strict plausibility modality, we 
can also define a unary “belief revision” modality *a, which in some sense 
internalizes the standard (binary) belief revision operator, by putting: 


*aP = PA [>a] P. 
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This gives us the following truth clause: 


s H *aP iff se sÈ. 


It is easy to see that *,P selects from any given information cell s(a) pre- 
cisely those states that satisfy agent a’s revised theory sť: 


*aP N s(a) = si. 


Recall that s? = Min<, s(a) N P is the conditional appearance of s to a 

given P, representing the agent’s “revised theory” (after revision with P) 

about s. This explains our interpretation: the proposition *,P is a complete 

description of the agent’s P-revised “theory” about the current state. 
Another interesting identity is the following: 


BPQ = Ka(*aP > Q). (2.8) 


In other words: Q is a conditional belief (given a condition P) iff it is a 
known consequence of the agent’s revised theory (after revision with P). 


Degrees of belief. Spohn’s “degrees of belief’ [49] were captured by 
Aucher [2] and van Ditmarsch [24] using logical operators B? P. Intuitively, 
0-belief B°P is the same as simple belief BaP; 1-belief BLP means that 
P is believed conditional on learning that not all the 0-beliefs are true etc. 
Formally, this can be introduced e.g., by defining by induction a sequence 
of appearance maps s} for all states s and natural numbers n: 


s? = Ming, s(a) , s? = Ming, (s(a) \ U si) 
i<n 


and defining 
sH BrP iff tH P for allt € st. 


A state s has degree of belief n if we have s € s?’. An interesting observation 
is that the finite degrees of belief BYP can be defined using the unary revision 
operator *,P and the knowledge operator Ka (and, as a consequence, they 
can be defined using the plausibility operator [>,]P and the knowledge 
operator). To do this, first put inductively: 


bs kT fhe = A \ =o) for alln > 1 


m<n 


and then put 
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“Strong belief’. Another important doxastic attitude can be defined in 
terms of knowledge and safe belief as: 


SbaP = BaP A K,(P > O,P). 


In terms of the plausibility order, it means that all the P-states in the 
information cell s(a) of s are bellow (more plausible than) all the non-P 
states in s(a) (and that, moreover, there are such P-states in s(a)). This 
notion is called “strong belief’ by Battigalli and Siniscalchi [13], while Stal- 
naker [51] calls it “robust belief’. Another characterization of strong belief 
is the following 


s = SbaQ iff: 
s H BaQ and s H BPQ for every P such that s = ~K, (P > ~Q). 


In other words: something is strong belief if it is believed and if this belief 
can only be defeated by evidence (truthful or not) that is known to contradict 
it. An example is the “presumption of innocence” in a trial: requiring the 
members of the jury to hold the accused as “innocent until proven guilty” 
means asking them to start the trial with a “strong belief’ in innocence. 


2.5 The logic of conditional beliefs 


The logic CDL (“conditional doxastic logic”) introduced in [8] is a logic of 
conditional beliefs, equivalent to the strongest logic considered in [19]. The 
syntax of CDL (without common knowledge and common belief operators!) 
is: 
p := p| >y | phy | Bre 

while the semantics is given by an interpretation map associating to each 
sentence y of CDL a doxastic proposition ||y||. The definition is by in- 
duction, in terms of the obvious compositional clauses (using the doxastic 
operators BPQ defined above). 

In this logic, knowledge and simple (unconditional) belief are derived 
operators, defined as abbreviations by putting Kay := B7’y, Bay := Bly 
(where T := ~(p A ~p) is some tautological sentence). 


Proof system. In addition to the rules and axioms of propositional logic, 
the proof system of CDL includes the following: 


Necessitation Rule: From F y infer + BYy. 
Normality: + B? (p > Y) > (B? > BYy) 


13 The logic in [8] has these operators, but for simplicity we decided to leave them aside 
in this presentation. 
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Truthfulness of Knowledge: | Kay > ọ 
Persistence of Knowledge: + Kay > Bey 
Full Introspection: + B?y > KaB? o, 
- =B} > KaB} o 
Success of Belief Revision: F Bọ 
Minimality of Revision: F ABP — (BEAYY > BP 9)) 


Proposition 2.4 (Completeness and Decidability). The above system is 
complete for MPMs (and so also for EPMs). Moreover, it is decidable and 
has the finite model property. 


Proof. The proof is essentially the same as in [19]. It is easy to see that 
the proof system above is equivalent to Board’s strongest logic in [19] (the 
one that includes axiom for full introspection), and that our models are 
equivalent to the “full introspective” version of the semantics in [19]. qQ.E.p. 


2.6 The logic of knowledge and safe belief 


The problem of finding a complete axiomatization of the logic of “defeasible 
knowledge” (safe belief) and conditional belief was posed as an open question 
in [19]. We answer this question here, by extending the logic CDL above to 
a complete logic KO of knowledge and safe belief. Since this logic can define 
conditional belief, it is in fact equivalent to the logic whose axiomatization 
was required in [19]. Solving the question posed there becomes in fact trivial, 
once we observe that the higher-order definition of “defeasible knowledge” 
in [52, 19] (corresponding to our identity (1.3) above) is in fact equivalent 
to our simpler, first-order definition of “safe belief”? as a Kripke modality. 


Syntax and semantics. The syntax of the logic KO is: 


yp := p| =y | erg | Cay | Kay 


while the semantics over plausibility models is given as for CDL, by induc- 
tively defining an interpretation map from sentences to doxastic proposi- 
tions, using the obvious compositional clauses. Belief and conditional belief 
are derived operators here, defined as abbreviations: 


Bry = Kap > Kaly ^ alp > ¥)), 
Bag := Blo, 


where Kay := ~K,7 is the Diamond modality for K, and T = =(pA-p) is 
some tautological sentence. So the logic KO is more expressive than CDL. 
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Proof system. In addition to the rules and axioms of propositional logic, 
the proof system for the logic KO includes the following: 


e the Necessitation Rules for both Ka and Oa; 


e the S5-axioms for Ka; 


e the S4-axioms for Oa; 


e K,P—O,P; 


Ka(P V OaQ) A Ka(QV Da P) > Ka P V KoQ. 


Theorem 2.5 (Completeness and Decidability). The logic KO is (weakly) 
complete with respect to MPMs (and so also with respect to EPMs). More- 
over, it is decidable and has the finite model property. 


Proof. A non-standard frame (model) is a structure (S, >a, ~a)a (together 
with a valuation, in the case of models) such that ~a are equivalence re- 
lations, >a are preorders, >, C ~a and the restriction of >a to each ~a- 
equivalence class is connected. For a logic with two modalities, Oa for >a 
and K, for the relation ~a, we can use well-known results in Modal Corre- 
spondence Theory to see that each of these semantic conditions corresponds 
to one of our modal axioms above. By general classical results on canon- 
icity and modal correspondence, we immediately obtain completeness for 
non-standard models. Finite model property for these non-standard models 
follows from the same general results. But every finite strict preorder rela- 
tion > is well-founded, and an MPM is nothing but a non-standard model 
whose strict preorders >, are well-founded. So completeness for (“stan- 
dard”) MPMs immediately follows. Then we can use Proposition 2.4 above 
to obtain completeness for EPMs. Finally, decidability follows, in the usual 
way, from finite model property together with completeness (with respect 
to a finitary proof system) and with the decidability of model-checking on 
finite models. (This last property is obvious, given the semantics.) Q.E.D. 


3 “Dynamic” Belief Revision 


The revision captured by conditional beliefs is of a static, purely hypothett- 
cal, nature. We cannot interpret BY as referring to the agent’s revised beliefs 
about the situation after revision; if we did, then the “Success” axiom 


+ Bey 
would fail for higher-level beliefs. To see this, consider a “Moore sentence” 


p := p^ Bap, 


14 See e.g., [18] for the general theory of modal correspondence and canonicity. 
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saying that some fact p holds but that agent a doesn’t believe it. The sen- 
tence y is consistent, so it may very well happen to be true. But agent a’s 
beliefs about the situation after learning that y was true cannot possibly 
include the sentence y itself: after learning this sentence, agent a knows p, 
and so he believes p, contrary to what y asserts. Thus, after learning y, 
agent a knows that ọ is false now (after the learning). This directly contra- 
dicts the Success axiom: far from believing the sentence after learning it to 
be true, the agent (knows, and so he correctly) believes that it has become 
false. There is nothing paradoxical about this: sentences may obviously 
change their truth values, due to our actions. Since learning the truth of a 
sentence is itself an action, it is perfectly consistent to have a case in which 
learning changes the truth value of the very sentence that is being learnt. 
Indeed, this is always the case with Moore sentences. Though not paradox- 
ical, the existence of Moore sentences shows that the “Success” axiom does 
not correctly describe a rational agent’s (higher-level) beliefs about what is 
the case after a new truth is being learnt. 

The only way to understand the “Success” axiom in the context of 
higher-level beliefs is to insist on the above-mentioned “static” interpre- 
tation of conditional belief operators BY’, as expressing the agent’s revised 
belief about how the state of the world was before the revision. 

In contrast, a belief update is a dynamic form of belief revision, meant 
to capture the actual change of beliefs induced by learning: the updated 
belief is about the state of the world as it is after the update. As noticed 
in [29, 6, 5], the original model does not usually include enough states to 
capture all the epistemic possibilities that arise in this way. While in the 
previous section the models were kept unchanged during the revision, all 
the possibilities being already there (so that both the unconditional and 
the conditional beliefs referred to the same model), we now have to allow 
for belief updates that change the original model. 

In [5], it was argued that epistemic events should be modeled in essen- 
tially the same way as epistemic states, and this common setting was taken 
to be given by epistemic Kripke models. Since in this paper we enriched our 
state models with doxastic plausibility relations to deal with (conditional) 
beliefs, it is natural to follow [5] into extending the similarity between ac- 
tions and states to this setting, thus obtaining (epistemic) action plausibility 
models. The idea of such an extension was first developed in [2] (for a differ- 
ent notion of plausibility model and a different notion of update product), 
then generalized in [24], where many types of action plausibility models and 
notions of update product, that extend the so-called Baltag-Moss-Solecki 
(BMS) update product from [6, 5], are explored. But both these works are 
based on a quantitative interpretation of plausibility ordinals (as “degrees of 
belief”), and thus they define the various types of products using complex 
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formulas of transfinite ordinal arithmetic, for which no intuitive justification 
is provided. 

In contrast, our notion of update product is a purely qualitative one, 
based on a simple and intuttive relational definition: the simplest way to de- 
fine a total pre-order on a Cartesian product, given total pre-orders on each 
of the components, is to use either the lexicographic or the anti-lexicographic 
order. We choose the second option, as the closest in spirit to the classi- 
cal AGM theory: it gives priority to the new, incoming information (i.e., 
to “actions” in our sense).!° We justify this choice by interpreting the ac- 
tion plausibility model as representing the agent’s “incoming” belief, i.e., 
the belief-updating event, which “performs” the update, by “acting” on the 
“prior” beliefs (as given in the state plausibility model). 


3.1 Action models 


An action plausibility model! (APM, for short) is a plausibility frame 
(©, <a)aca together with a precondition map pre : X — Prop, associating 
to each element of X some doxastic proposition pre,. We call the elements 
of & (basic) doxrastic actions (or “events” ), and we call pre, the precondition 
of action ø. The basic actions ø € X are taken to represent deterministic 
belief-revising actions of a particularly simple nature. Intuitively, the pre- 
condition defines the domain of applicability of action ø: it can be executed 
on a state s iff s satisfies its precondition. The relations <a give the agents’ 
beliefs about which actions are more plausible than others. 

To model non-determinism, we introduce the notion of epistemic pro- 
gram. A doxastic program over a given action model % (or -program, for 
short) is simply a set IT C £ of doxastic actions. We can think of doxastic 
programs as non-deterministic actions: each of the basic actions y €T is a 
possible “deterministic resolution” of I. For simplicity, when T = {y} is a 
singleton, we ambiguously identify the program I with the action y. 

Observe that %-programs I C © are formally the “dynamic analogues” 
of S-propositions P C S. So the dynamic analogue of the conditional dox- 
astic appearance sẹ (representing agent a’s revised theory about state s, 
after revision with proposition P) is the set ot. 


Interpretation: beliefs about changes encode changes of beliefs. 
The name “doxastic actions” might be a bit misleading, and from a philo- 
sophical perspective Johan van Benthem’s term “doxastic events” seems 
more appropriate. The elements of a plausibility model do not carry infor- 
mation about agency or intentionality and cannot represent “real” actions 
in all their complexity, but only the doxastic changes induced by these ac- 
tions: each of the nodes of the graph represents a specific kind of change 


15 This choice can be seen as a generalization of the so-called “mazximal-Spohn” revision. 
16 Van Benthem calls this an “event model”. 
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of beliefs (of all the agents). As in [5], we only deal here with pure “belief 
changes”, i.e., actions that do not change the “ontic” facts of the world, 
but only the agents’ beliefs.!” Moreover, we think of these as deterministic 
changes: there is at most one output of applying an action to a state.!® 
Intuitively, the precondition defines the domain of applicability of o: this 
action can be executed on a state s iff s satisfies its precondition. The 
plausibility pre-orderings <a give the agents’ conditional beliefs about the 
current action. But this should be interpreted as beliefs about changes, that 
encode changes of beliefs. In this sense, we use such “beliefs about actions” 
as a way to represent doxastic changes: the information about how the 
agent changes her beliefs is captured by our action plausibility relations. So 
we read o <a o’ as saying that: if agent a is informed that either o or o’ 
is currently happening, then she cannot distinguish between the two, but 
she believes that o is in fact happening. As already mentioned, doxastic 
programs [I C X represent non-deterministic changes of belief. Finally, for 
an action c and a program T, the program o! represents the agent’s revised 
theory (belief) about the current action o after “learning” that (one of the 
deterministic resolutions y in) T is currently happening. 


Example 3.1 (Private “Fair-Game” Announcements). Let us consider the 
action that produced the situation represented in Example 2.2 above. In 
front of Alice, Bob looked at the coin, in such a way that (it was common 
knowledge that) only he saw the face. In the DEL literature, this is some- 
times known as a “fair game” announcement: everybody is commonly aware 
that an insider (or a group of insiders) privately learns some information. 
It is “fair” since the outsiders are not “deceived” in any way: e.g., in our 
example, Alice knows that Bob looks at the coin (and he knows that she 
knows etc.). In other words, Bob’s looking at the coin is not an “illegal” 
action, but one that obeys the (commonly agreed) “rules of the game”. To 
make this precise, let us assume that this is happening in such a way that 
Alice has no strong beliefs about which of the two possible actions (Bob- 
seeing-Heads-up and Bob-seeing-Tails-up) is actually happening. Of course, 
we assumed that before this, she already believed that the coin lies Heads 
up, but apart from this we now assume that the way the action (of “Bob 
looking”) is happening gives her no indication of what face he is seeing. 
We represent these actions using a two-node plausibility model Xə (where 
as in the case of state models we draw arrows for the converse plausibility 


17 We stress this is a minor restriction, and it is very easy to extend this setting to 
“ontic” actions. The only reason we stick with this restriction is that it simplifies the 
definitions, and that it is general enough to apply to all the actions we are interested 
here, and in particular to all communication actions. 

18 As in [5], we will be able to represent non-deterministic actions as sums (unions) of 
deterministic ones. 
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relations >a, disregarding all the loops): 


H Q T 
e e 


Example 3.2 (Fully Private Announcements). Let us consider the action 
that produced the situation represented in Example 2.3 above. This was the 
action of Bob taking a peek at the coin, while Alice was away. Recall that we 
assumed that Alice believed that nothing was really happening in her absence 
(since she assumed Bob was playing by the rules), though obviously she 
didn’t know this (that nothing was happening). In the DEL literature, this 
action is usually called a fully private announcement: Bob learns which face 
is up, while the outsider Alice believes nothing of the kind is happening. To 
represent this, we consider an action model X; consisting of three “actions” : 
the actual action ø in which Bob takes a peek and sees the coin lying Heads 
up; the alternative possible action p is the one in which Bob sees the coin 
lying Tails up; finally, the action 7 is the one in which “nothing is really 
happening” (as Alice believes). The plausibility model 43 for this action is: 


He T 


NA 


Here, the action ø is the one in the upper left corner, having precondition 
H: indeed, this can happen iff the coin is really lying Heads up; similarly, 
the action p in the upper right corner has precondition T, since it can only 
happen iff the coin is Tails up. Finally, the action 7 is the lower one, having 
as precondition the “universally true” proposition T: indeed, this action can 
always happen (since in it, nothing is really happening!). The plausibility 
relations reflect the agents’ beliefs: in each case, both Bob and Charles 
know exactly what is happening, so their local plausibility relations are the 
identity (and thus we draw no arrows for them). Alice believes nothing is 
happening, so 7 is the most plausible action for her (to which all her arrows 
are pointing); so she keeps her belief that H is the case, thus considering o 
as more plausible than p. 


Examples of doxastic programs. Consider the program I = {o,p} C 
X3 over the action model “3 from Example 3.2. The program I represents 
the action of “Bob taking a peek at the coin”, without any specification of 
which face he is seeing. Although expressed in a non-deterministic manner 
(as a collection of two possible actions, ø and p), this program corresponds 
is in fact deterministic, since in each possible state only one of the actions 
g or p can happen: there is no state satisfying both H and T. The whole 
set % gives another doxastic program, one that is really non-deterministic: 
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it represents the non-deterministic choice of Bob between taking a peek and 
not taking it. 

Appearance of actions and their revision: Examples. As an example 
of an agent’s “theory” about an action, consider the appearance of action p 
to Alice: pa = {r}. Indeed, if p happens (Bob taking a peek and sees the 
coin is Tails up), Alice believes that 7 (i.e., nothing) is happening: this is 
the “apparent action”, as far as Alice is concerned. As an example of a 
“revised theory” about an action, consider the conditional appearance pF 
of p to Alice given the program I = {o,p} introduced above. It is easy 
to see that we have pl = {a}. This captures our intuitions about Alice’s 
revised theory: if, while p was happening, she were told that Bob took a 
peek (i.e., she’d revise with I’), then she would believe that he saw the coin 
lying Heads up (i.e., that o happened). 


Example 3.3 (Successful Lying). Suppose now that, after the previous 
action, i.e., after we arrived in the situation described in Example 2.3, Bob 
sneakily announces: “I took a peek and saw the coin was lying Tails up”. 
We formalize the content of this announcement as K,T, i.e., saying that 
“Bob knows the coin is lying Tails up”. This is a public announcement, but 
not a truthful one (though it does convey some truthful information): it is 
a lie! We assume it is in fact a successful lie: it is common knowledge that, 
even after Bob admitted having taken a peek, Alice still believes him. This 
action is given by the left node in the following model ¥4: 


-K,T a -KT 
e e 


3.2 The action-priority update 


We are ready to define our update operation, representing the way an action 
from a (action) plausibility model X = (X, <a, pre)aca “acts” on an input- 
state from a given (state) plausibility model S = (S, <a, ||-||)ae.4. We denote 
the updated state model by S ® ©, and call it the update product of the two 
models. The construction is similar to a point to the one in [6, 5], and thus 
also somewhat similar to the ones in [2, 24]. In fact, the set of updated 
states, the updated valuation and the updated indistinguishability relation 
are the same in these constructions. The main difference lies in our definition 
of the updated plausibility relation, via the Action Priority Rule. 


3.2.1 Updating single-agent models: the anti-lexicographic 
order 

To warm up, let us first define the update product for the single-agent 

case. Let S = (S,<,||-||) be a single-agent plausibility state model and let 

X = (£, <, pre) be a single-agent plausibility action model. 
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We represent the states of the updated model S & © as pairs (s,0) of 
input-states and actions, i.e., as elements of the Cartesian product S x X. 
This reflects that the basic actions in our action models are assumed to be 
deterministic: For a given input-state and a given action, there can only be 
at most one output-state. More specifically, we select the pairs which are 
consistent, in the sense that the input-state satisfies the precondition of the 
action. This is natural: the precondition of an action is a specification of 
its domain of applicability. So the set of states of S ® © is taken to be 


SE := {(s,0): s Hs pre(o)}. 


The updated valuation is essentially given by the original valuation from the 
input-state model: For all (s,o) € S Q £, we put (s,0) E p iff s E p. This 
“conservative” way to update the valuation expresses the fact that we only 
consider here actions that are “purely doxastic”, i.e., pure “belief changes”, 
that do not affect the ontic “facts” of the world (captured here by atomic 
sentences). 

We still need to define the updated plausibility relation. To motivate 
our definition, we first consider two examples: 


Example 3.4 (A Sample Case). Suppose that we have two states s,s’ € 
S such that s < s, s H =P, s = P. This means that, if given the 
supplementary information that the real state is either s or s’, the agent 
believes =P: 


“SR P 
e e 


Suppose then an event happens, in whose model there are two actions a, 0’ 
such that o > o’, pre, = —P, pre, = P. In other words, if given the 
information that either ø or o’ is happening, the agent believes that a’ is 
happening, i.e., she believes that P is learnt. This part of the model behaves 
just like a soft public announcement of P: 


=P >P 
e e 


Naturally, we expect the agent to change her belief accordingly, i.e., her 
updated plausibility relation on states should now go the other way: 
=P -P 
e e 
Example 3.5 (A Second Sample Case). Suppose the initial situation was 


the same as above, but now the two actions 0,0’ are assumed to be equi- 
plausible: ø = o’. This is a completely unreliable announcement of P, in 
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which the veracity and the falsity of the announcement are equally plausible 
alternatives: 


“PeP 
e e 


In the AGM paradigm, it is natural to expect the agents to keep their 
original beliefs unchanged after this event: 


aP. P 
e e 


The anti-lexicographic order. Putting the above two sample cases to- 
gether, we conclude that the updated plausibility relation should be the 
anti-lexicographic preorder relation induced on pairs (s,a0) € S x £ by the 
preorders on S and on ®, i.e.: 


(s,o) <(s',o’) iff: either o <o’, or else o S o’ ands < s. 


In other words, the updated plausibility order gives “priority” to the action 
plausibility relation, and apart from this it keeps as much as possible the old 
order. This reflects our commitment to an AGM-type of revision, in which 
the new information has priority over old beliefs. The “actions” represent 
here the “new information”, although (unlike in AGM) this information 
comes in dynamic form (as action plausibility order), and so it is not fully 
reducible to its propositional content (the action’s precondition). In fact, 
this is a generalization of one of the belief-revision policies encountered in 
the literature (the so-called “mazimal-Spohn revision”). But, in the context 
of our qualitative (conditional) interpretation of plausibility models, we will 
argue below that this is essentially the only reasonable option. 


3.2.2 Updating multi-agent models: the general case 

In the multi-agent case, the construction of the updated state space and 
updated valuation is the same as above. But for the updated plausibility 
relation we need to take into account a third possibility: the case when either 
the initial states or the actions are distinguishable, belonging to different 
information cells. 


Example 3.6 (A Third Sample Case). Suppose that we have two states 
s,s’ E€ S such that s E AP, s’ E P, but s %a s’ are distinguishable (i.e., 
non-comparable): 


=P P 
e e 

This means that, if given the supplementary information that the real 
state is either s or s’, the agent immediately knows which of the two is 
the real states, and thus she knows whether P holds or not. It is obvious 
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that, after any of the actions considered in the previous two examples, a 
perfect-recall agent will continue to know whether P held or not, and so the 
output-states after o and o’ will still be distinguishable (non-comparable). 


The “Action-Priority” Rule. Putting this together with the other sam- 
ple cases, we obtain our update rule, in full generality: 


(8,0) <a (s’,0’) iff either o <a 0’ and s ~a 8’, or else o Sa 0’ and s Sa 8’ 


We regard this construction as the most natural analogue in a belief- 
revision context of the similar notion in [5, 6]. Following a suggestion of 
Johan van Benthem, we call this the Action-Priority Update Rule. 


Sanity check: Examples 2.2 and 2.3 revisited. To check the correct- 
ness of our update operation, take first the update product S ® No of the 
model S in Example 2.1 from the previous section with the action model 
Xə in Example 3.1 from the previous section. As predicted, the resulting 
state model is isomorphic to the model W from Example 2.2. Similarly, if 
3 is the action model from Example 3.2, then we can see that the product 
S ® X£; is isomorphic to the state model S’ from Example 2.3. 


“In-sanity check”: Successful lying. Applying the action model X, in 
Example 3.3, representing the “successful lying” action, to the state model 
S’ from Example 2.3, we obtain indeed the intuitively correct output of 
“successful lying”, namely the following model S’ ® Ny: 


H Q >T 


eH ° TT 
a,b 


Interpretation. As its name makes explicit, the Action-Priority Rule gives 
“priority” to the action plausibility relation. This is not an arbitrary choice, 
but it is motivated by our specific interpretation of action models, as embod- 
ied in our Motto above: beliefs about changes (i.e., the action plausibility 
relations) are nothing but ways to encode changes of belief (i.e., reversals 
of the original plausibility order). So the (strict) order on actions encodes 
changes of order on states. The Action-Priority Rule is a consequence of 
this interpretation: it just says that a strong plausibility order o <q a’ 
on actions corresponds indeed to a change of ordering, (from whatever the 
ordering was) between the original (indistinguishable) input-states s ~a s’, 
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to the order (s,0) <a (s’,o’) between output-states; while equally plausible 
actions o =, a’ will leave the initial ordering unchanged: (s,0) <a (s’,0’) 
iff s <, s’. Giving priority to action plausibility does not in any way mean 
that the agent’s belief in actions is stronger than her belief in states; it just 
captures the fact that, at the time of updating with a given action, the belief 
about the action is what is actual, it is the current belief about what is going 
on, while the beliefs about the input-states are in the past.'® 

In a nutshell: the doxastic action is the one that changes the initial 
doxastic state, and not vice-versa. The belief update induced by a given 
action is nothing but an update with the (presently) believed action. If the 
believed action o requires the agent to revise some past beliefs, then so be 
it: this is the whole point of believing ø, namely to use it to revise one’s past 
beliefs. For example, in a successful lying, the action plausibility relation 
makes the hearer believe that the speaker is telling the truth; so she’ll accept 
this message (unless contradicted by her knowledge), and change her past 
beliefs appropriately: this is what makes the lying successful. 


Action-priority update generalizes product update. Recall the def- 
inition of the epistemic indistinguishability relation ~a in a plausibility 
model: s ~a 8s’ iff either s <a s’ or s’ <q s. It is easy to see that the 
Action Priority Update implies the familiar update rule from [6, 5], known 
in Dynamic Epistemic Logic as the “product update”: 


(8,0) ~a (s',0") iff s ~as’ and o ~ao. 


Program transitions. For every state model S, every program [ C X 


over an action model © induces a transition relation gC S x (S£) from 
S to S @ &, given by: 


s Ds (s',y) if s=s',(s,y)€S@uUandyer. 


3.3 Simulating various belief-revision policies 


We give here three examples of multi-agent belief-revision policies that can 
be simulated by our product update: truthful public announcements of “hard 
facts”, “lexicographic update” and “conservative upgrade”. They were all 
introduced by van Benthem in [14], as multi-agent versions of revision op- 
erators previously considered by Rott [45] and others. 


Public announcements of “hard facts”. A truthful public announce- 
ment! P of some “hard fact” P is not really about belief revision, but about 
the learning of certified true information: it establishes common knowledge 


19 Of course, at a later moment, the above-mentioned belief about action (now belonging 
to the past) might be itself revised. But this is another, future update. 
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that P was the case. This is the action described in [14] as (public) “belief 
change under hard facts”. As an operation on models, this is described in 
[14] as taking any state model S and deleting all the non-P states, while 
keeping the same indistingutshability and plausibility relations between the 
surviving states. In our setting, the corresponding action model consists of 
only one node, labeled with P. It is easy to see that the above operation on 
models can be exactly “simulated” by taking the anti-lexicographic product 
update with this one-node action model. 


Public announcements of “soft facts”: The “lexicographic up- 
grade”. To allow for “soft” belief revision, an operation {}P was introduced 
in [14], essentially adapting to public announcements the ‘lexicographic’ pol- 
icy for belief revision described in [45]. This operation, called “lexicographic 
update” consists of changing the current plausibility order on any given state 
model as follows: every P-world becomes “better” (more plausible) than all 
=P -worlds in the same information cell, and within the two zones (P and 
aP ), the old ordering remains. In our setting, this action corresponds to 
the following local plausibility action model: 
a,b,c,-- 


=P >P 
e e 


Taking the anti-lexicographic update product with this action will give an 
exact “simulation” of the lexicographic upgrade operation. 


“Conservative upgrade”. The operation TP of “conservative upgrade”, 
also defined in [14], changes any model as follows: in every information cell, 
the best P-worlds become better than all the worlds in that cell (i.e., in every 
cell the most plausible P-states become the most plausible overall in that 
cell), and apart from that, the old order remains. In the case of a system 
with only one agent, it is easy to see that we have JP = ff(*aP), where xa 
is the unary “revision modality” introduced in the previous section. In the 
case of a set A = {1,--- ,n} with n > 1 agents, we can simulate ÎP using 
a model with 2” actions {17P }rca, with 


prer, p = \\ *;P N \ TK P, 
icI jgI 
TıP <k TJP iff JN{k} CI. 


3.4 Operations on doxastic programs 


First, we introduce dynamic modalities, capturing the “weakest precondi- 
tion” of a program T. These are the natural analogues of the PDL modalities 


ius y T 
for our program transition relations — between models. 
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Dynamic modalities. Let © be some action plausibility model and T C X 
be a doxastic model over X. For every doxastic proposition P, we define a 
doxastic proposition [[']P given by 


(I]P)s := [Ès]Ps = {s € S: YtE SQE (s Sgt > t Ksgu P)}. 


For basic doxastic actions o € X, we define the dynamic modality [o] via the 
above-mentioned identification of actions ø with singleton programs {co}: 


(IoJP)s := ({o}]P)g ={s eS: if(s,c) €S@ then (s,0) € Psgs}. 


The dual (Diamond) modalities are defined as usually: (T}P :=—[T]=P. 
We can now introduce operators on doxastic programs that are the ana- 
logues of the regular operations of PDL. 


Sequential composition. The sequential composition X; A of two action 
plausibility models © = (£, <a, pre), A = (A, <a, pre) is defined as follows: 


e the set of basic actions is the Cartesian product © x A 
e the preconditions are given by pre, 5) := (7) pres 


e the plausibility order is given by putting (0,6) <a (o’, 6’) iff: 
either o <a a’ and 6 ~a 0’, or else o Xa o’ and 6 <a 6’. 


We think of (0,6) as the action of performing first o then 6, and thus 
use the notation 


o; ô := (0,6). 


We can extend this notation to doxastic programs, by defining the sequential 
composition of programs T C X and A C A to be a program T; A C X; A 
over the action model X; A, given by: 


T;A:={(7, A): y ET AEG A}. 
It is easy to see that this behaves indeed like a sequential composition: 


Proposition 3.7. For every state plausibility model S, action plausibility 
models © and A, and programs I C X, A C A, we have the following: 


1. The state plausibility models (S @ £) @ A and S ® (X; A) are iso- 
morphic, via the canonical map F : (S 8 ©) @ A > S 8 (X; A) given 


by 
F(((s,¢), 9) ) = (s, (a, ô)). 
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2. The transition relation for the program I; A is the relational composi- 
tion of the transition relations for I and for A and of the isomorphism 
map F: 


s Log s’ iff there exist w,t € S Q E such that 


s L w SE t and F(t) = 8’. 


Union (non-deterministic choice). If © = (X,<,,pre) and A = 
(A, </,, pre’) are two action plausibility models, their disjoint union SUA 
is simply given by taking as set of states the disjoint union X U A of the two 
sets of states, taking as plausibility order the disjoint union <, Li </, and 
as precondition map the disjoint union prepre’ of the two precondition 
maps. IfI C X and A C A are doxastic programs over the two models, we 
define their union to be the program over the model 4 U A given by the 
disjoint union I U A of the the sets of actions of the two programs. 

Again, it is easy to see that this behaves indeed like a non-deterministic 
choice operator: 


Proposition 3.8. Let i : E — MUA and i2 : A > MUA be the two 
canonical injections. Then the following are equivalent: 


TUA , 
es—gs 


e there exists t such that: 


either s s t and i(t) = s’, or else s 4st and ig(t) = 8’. 


Other operators. Arbitrary unions ||,T; can be similarly defined, and 
then one can define iteration IT* := ||, T° (where T° =!T and r+! =T;I"). 


3.5 The laws of dynamic belief revision 


The “laws of dynamic belief revision” are the fundamental equations of 
Belief Dynamics, allowing us to compute future doxastic attitudes from past 
ones, given the doxastic events that happen in the meantime. In modal 
terms, these can be stated as “reduction laws” for inductively computing 
dynamic modalities [[]P, by reducing them to modalities [[’]P’ in which 
either the propositions P’ or the programs I” have lower complexity. 

The following immediate consequence of the definition of [T]P allows us 
to reduce modalities for non-deterministic programs I to the ones for their 
deterministic resolutions y € T: 
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Deterministic Resolution Law. For every program I C X, we have 
rP = ADP. 
yer 
So, for our other laws, we can restrict ourselves to basic actions in X. 
The Action-Knowledge Law. For every action ø € X}, we have: 
lo] KaP = pre, > \ K,{o'|P. 
o'ngo 


This Action-Knowledge Law is essentially the same as in [6, 5]: a proposition 
P will be known after a doxastic event iff, whenever the event can take place, 
it is known that P will become true after all events that are indistinguishable 
from the given one. 


The Action-Safe-Belief Law. For every action ø € X, we have: 


[o]O,P = pre, > VAN Kalo']P A \ alo’ JP. 


wt 
a'<aQ o" %a0 


This law embodies the essence of the Action-Priority Rule: a proposition P 
will be safely believed after a doxastic event iff, whenever the event can take 
place, it is known that P will become true after all more plausible events 
and in the same time it is safely believed that P will become true after all 
equi-plausible events. 

Since we took knowledge and safe belief as the basis of our static logic 
KO, the above two laws are the “fundamental equations” of our theory of 
dynamic belief revision. But note that, as a consequence, one can obtain de- 
rived laws for (conditional) belief as well. Indeed, using the above-mentioned 
characterization of conditional belief in terms of K and O, we obtain the 
following: 


The Derived Law of Action-Conditional-Belief. For every action 
o € X}, we have: 


(]BPQ = pre, > V ( N Rah) P A A -Ralp A BS? [oh ]Q) | 


rcs yer y'gr 


This derived law, a version of which was first introduced in [10] (where it 
was considered a fundamental law), allows us to predict future conditional 
beliefs from current conditional beliefs. 

To explain the meaning of this law, we re-state it as follows: For every 
s € S and o € ®, we have: 


sH [o]BEQ if spre, > BYP T]Q, 
where T = {7€ X: s Hs Kaly)P}. 
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It is easy to see that this “local” (state-dependent) version of the reduc- 
tion law is equivalent to the previous (state-independent) one. The set T 
encodes the extra information about the current action that is given to the 
agent by the context s and by the post-condition P; while øf is the ac- 
tion’s post-conditional contextual appearance, i.e., the way it appears to the 
agent in the view of this extra-information I. Indeed, a given action might 
“appear” differently in a given context (i.e., at a state s) than it does in 
general: the information possessed by the agent at the state s might imply 
the negation of certain actions, hence their impossibility; this information 
will then be used to revise the agent’s beliefs about the actions, obtaining 
her contextual beliefs. Moreover, in the presence of further information (a 
“post-condition” P), this appearance might again be revised. The “post- 
conditional contextual appearance” is the result of this double revision: the 
agent’s belief about action o is revised with the information given to her 
by the context s and the post-condition P. This information is encoded in 
a set IT = {y € È : s Fg Ka(y)P} of “admissible” actions: the actions for 
which the agent considers epistemically possible (at s) that they can be per- 
formed and they can achieve the post-condition P. The “post-conditional 
contextual appearance” of of action o captures the agent’s revised theory 
about o after revision with the relevant information I. 

So the above law says that: the agent’s future conditional beliefs |o] BE 
can be predicted, given that action o happens, by her current conditional 


Tr 
beliefs BYP [oT] about what will be true after the apparent action of (as 


it appears in the given context and in the view of the given post-condition 
P), beliefs conditioned on the information (lo! YP) that the apparent action 
ol actually can lead to the fulfillment of the post-condition P. 


Special cases. As special cases of the Action-Conditional-Belief Law, we 
can derive all the reduction laws in [14] for (conditional) belief after the 
events ! P, tP and TP: 


[P]BoR =P — BP PQ piR, 

(NPIBOR = (KP [MP]Q A BEF! EPIR) v (KT IMPIQ A BEA PPIR) , 
(TPIBOR = (BP [TP]Q ^ BETFISTPIR) v (“BP TPIQ A BUPIOIPIR) , 
where 


KiQ:=K.(P-Q), KiQ:=7KP-Q, BFQ:=-BF-Q. 


Laws for other doxastic attitudes. The equi-plausibility modality be- 
haves dynamically “almost” like knowledge, while the strict plausibility 
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modality behaves like safe belief, as witnessed by the following laws: 


lo][S]P = pre > A Eo, 
lo][>a]P = preo > N KalPa AN Pallo”]P 


From these, we can derive laws for all the other doxastic attitudes above. 


3.6 The logic of doxastic actions 


The problem of finding a general syntax for action models has been tack- 
led in various ways by different authors. Here we use the action-signature 
approach from [5]. 


Signature. A doxastic action signature is a finite plausibility frame X, 
together with an ordered list without repetitions (o1,...,0n) of some of the 
elements of X. The elements of © are called action types. A type ø is called 
trivial if it is not in the above list. 


Example 3.9. The “hard” public announcement signature HardPub is a 
singleton frame, consisting of one action type !, identity as the order relation, 
and the list (!). 

The “soft” public announcement signature SoftPub is a two-point frame, 
consisting of types ft and J}, with 4 <a ft for all agents a, and the list (ft, 4). 

Similarly, one can define the signatures of fully private announcements 
with n alternatives, private “fair-game” announcements, conservative up- 
grades etc. As we will see below, there is no signature of “successful (public) 
lying”: public lying actions fall under the type of “soft” public announce- 
ments, so they are generated by that signature. 


Languages. For each action signature (X, (o1,...,0n)), the language 
L(=) consists of a set of sentences y and a set of program terms m, de- 
fined by simultaneous recursion: 


p = P| el ere | Kay | Gay | [r]o 
T = O1... Pn | TUT |T; 


where p € Ọ, a E€ A, o € ÈX, and ogi ...pn is an expression consisting of o 
and a string of n sentences, where n is the length of the list (01, ..., On). 


Syntactic action model. The expressions of the form o¢ are called basic 
programs. The preorders on & induce in a natural way preorders on the 
basic programs in L(%): 


og La ow iff o <a d'and =w. 
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The given listing can be used to assign syntactic preconditions for basic 
programs, by putting: pre,,g := Pi, and pre,g := T (the trivially true 
sentence) if ø is not in the listing. Thus, the basic programs of the form 
og form a “syntactic plausibility model” XØ; i.e., every given interpretation 
||-|| : LC) — Prop of sentences as doxastic propositions will convert this 


‘ 


syntactic model into a “real” (semantic) plausibility model, called Sls. 


Action models induced by a signature. For a given signature &, let 
(o1,---;On) be its list of non-trivial types, and let P= (Pi,...,Pn) bea 
matching list of doxastic propositions. The action model generated by the 
signature X and the list of propositions P is the model =P, having © as its 
underlying action frame and having a precondition map given by: pre,, = 
P,, for non-trivial types o;; and pre, = T (the trivially true proposition), 
for trivial types ø. When referring to ø as an action in =P, we will denote 
it by oP, to distinguish it from the action type o € X. 

We can obviously extend this construction to sets of action types: given 
a signature © and a list P = (P1,..., Pn), every set IT C X gives rise to a 
doxastic program [P := {oP : o € X} CUP. 


Example 3.10. The action model of a hard public announcement !P is 
generated as !(P) by the hard public announcement signature HardPub = {!} 
and the list (P). Similarly, the action model SoftPub(P) generated by the 
soft public announcement signature SoftPub and a list (P, Q) of two proposi- 
tions consists of two actions {}(P, Q) and (P, Q), with (P, Q) <a (P,Q), 
prey(p,q) = P and prey(p.q) = Q: 


a,b,c 


Q b,c, 
e 


This represents an event during which all agents share a common belief that 
P is announced; but they might be wrong and maybe Q was announced in- 
stead. However, it is common knowledge that either P or Q was announced. 

Successful (public) lying Lie P (by an anonymous agent, falsely announc- 
ing P) can now be expressed as LieP := \}(P,-P). The truthful soft an- 
nouncement is TrueP := {}(P,-P). Finally, the soft public announce- 
ment (lexicographic update) {}P, as previously defined, is given by the non- 
deterministic union fP := True P U LieP. 


Semantics. We define by simultaneous induction two interpretation maps, 
one taking sentences y into doxastic propositions ||y|| € Prop, the second 
taking program terms 7 into doxastic programs ||7|| over some plausibility 
frames. The inductive definition uses the obvious semantic clauses. For 
programs: ||o@]| is the action o||y|| (or, more exactly, the singleton program 


{elll} over the frame ¥l|y|[), lru r'| = [lr] Ula’, lls = [ab I'l 
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For sentences: ||p|| is as given by the valuation, ||-y]| := —||yI|, l A || := 
lell A lel, Kagll = Kallel Bagel] = allel, Halol = [lal] llel 


Proof system. In addition to the axioms and rules of the logic KO, the 
logic L(%) includes the following Reduction Axioms: 


[a]jp = pre, > p 
[azy > prey > [aly 
laly ny) = prea > [aly A [a] 
la] Kap © prea > N Kalole 
[a] ap © Prea 7” \ K,[a’]y A \ ala” |p 


kurip a kien ele g 
[m;n] = [r] [r] 


where p is any atomic sentence, 7,7’ are program terms, a is a basic pro- 
gram term in L(%), pre is the syntactic precondition map defined above, 
and ~a, <a; a are respectively the (syntactic) epistemic indistinguishabil- 
ity, the strict plausibility order and the equi-plausibility relation on basic 
programs. 


Theorem 3.11. For every signature X, the above proof system for the 
dynamic logic L(%) is complete, decidable and has the finite model property. 
In fact, this dynamic logic has the same expressive power as the “static” 
logic KO of knowledge and safe belief. 


Proof (Sketch). The proof is similar to the ones in [5, 6, 25]. We use the 
reduction laws to inductively simplify any formula until it is reduced to a 
formula of the KO-logic, then use the completeness of the KU logic. Note 
that this is not an induction on subformulas, but (as in [6]) on an appropriate 
notion of “complexity” ordering of formulas. Q.E.D. 


4 Current and Future Work, Some Open Questions 


In our papers [11, 12], we present a probabilistic version of the theory devel- 
oped here, based on discrete (finite) Popper-Renyi conditional probability 
spaces (allowing for conditionalization on events of non-zero probability, 
in order to cope with non-trivial belief revisions). We consider subjective 
probability to be the proper notion of “degree of belief”, and we investigate 
its relationship with the qualitative concepts developed here. We develop a 
probabilistic generalization of the Action Priority Rule, and show that the 
logics presented above are complete for the (discrete) conditional probabilis- 
tic semantics. 
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We mention here a number of open questions: (1) Axiomatize the full 
(static) logic of doxastic attitudes introduced in this paper. It can be easily 
shown that they can all be reduced to the modalities Ka, [>a] and [Fa]. 
There are a number of obvious axioms for the resulting logic K [>] [=] (note 
in particular that [>] satisfies the Godel-Léb formula!), but the completeness 
problem is still open. (2) Axiomatize the logic of common safe belief and 
common knowledge, and their dynamic versions. More generally, explore the 
logics obtained by adding fixed points, or at least “epistemic regular (PDL- 
like) operations” as in [15], on top of our doxastic modalities. (3) Investigate 
the expressive limits of this approach with respect to belief-revision policies: 
what policies can be simulated by our update? (4) Extend the work in [11, 
12], by investigating and axiomatizing doxastic logics on infinite conditional 
probability models. (5) Extend the logics with quantitative (probabilistic) 
modal operators B? „Q (or Da zQ) expressing that the degree of conditional 
belief in Q given P {ot the degree of safety of the belief in Q) is at least x. 
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Abstract 

We consider strategic-form games with ordinal payoffs and provide a 
syntactic analysis of common belief/knowledge of rationality, which 
we define axiomatically. Two axioms are considered. The first says 
that a player is irrational if she chooses a particular strategy while 
believing that another strategy is better. We show that common belief 
of this weak notion of rationality characterizes the iterated deletion 
of pure strategies that are strictly dominated by pure strategies. The 
second axiom says that a player is irrational if she chooses a particular 
strategy while believing that a different strategy is at least as good 
and she considers it possible that this alternative strategy is actually 
better than the chosen one. We show that common knowledge of this 
stronger notion of rationality characterizes the restriction to pure 
strategies of the iterated deletion procedure introduced by Stalnaker 
(1994). Frame characterization results are also provided. 


1 Introduction 


The notion of rationalizability in games was introduced independently by 
Bernheim [2] and Pearce [16]. A strategy of player i is said to be ratio- 
nal if it maximizes player 7’s expected payoff, given her probabilistic beliefs 
about the strategies used by her opponents; that is, if it can be justified 
by some beliefs about her opponents’ strategies. If player 7, besides being 
rational, also attributes rationality to her opponents, then she must only 
consider as possible strategies of her opponents that are themselves justi- 
fiable. If, furthermore, player i believes that her opponents believe that 
she is rational, then she must believe that her opponents justify their own 
choices by only considering those strategies of player 7 that are justifiable, 
and so on. The strategies of player i that can be justified in this way are 
called rationalizable. Rationalizability was intended to capture the notion 
of common belief of rationality. Bernheim and Pearce showed that a strat- 
egy is rationalizable if and only if it survives the iterated deletion of strictly 
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dominated strategies.! They captured the notion of common belief of ratio- 
nality only informally, that is, without making use of an epistemic frame- 
work. The first epistemic characterization of rationalizability was provided 
by Tan and Werlang [18] using a universal type space, rather than Kripke 
structures (Kripke [13]). A characterization of common belief of rationality 
using probabilistic Kripke structures was first provided by Stalnaker [17], 
although it was implicit in Brandenburger and Dekel [8]. Stalnaker also in- 
troduced a new, stronger, notion of rationalizability—which he called strong 
rationalizability—and showed that it corresponds to an iterated deletion 
procedure which is stronger than the iterated deletion of strictly dominated 
strategies. Stalnaker’s approach is entirely semantic and uses the same no- 
tion of Bayesian rationality as Bernheim and Pearce, namely expected pay- 
off maximization. This notion presupposes that the players’ payoffs are von 
Neumann-Morgenstern payoffs. In contrast, in this paper we consider the 
larger class of strategic-form games with ordinal payoffs. Furthermore, we 
take a syntactic approach and define rationality axiomatically. We consider 
two axioms. 

The first axiom says that a player is irrational if she chooses a partic- 
ular strategy while believing that another strategy of hers is better. We 
show that common belief of this weak notion of rationality characterizes 
the iterated deletion of strictly dominated pure strategies. Note that, in 
the Bayesian approach based on von Neumann-Morgenstern payoffs, it can 
be shown (see Pearce [16] and Brandenburger and Dekel [8]) that a pure 
strategy s; of player i is a best reply to some (possibly correlated) beliefs 
about the strategies of her opponents if and only if there is no mized strat- 
egy of player i that strictly dominates s;. The iterated deletion of strictly 
dominated strategies in the Bayesian approach thus allows the deletion of 
a pure strategy that is dominated by a mized strategy, even though it may 
not be dominated by another pure strategy. Since we take a purely ordinal 
approach, the iterated deletion procedure that we consider only allows the 
removal of strategies that are dominated by pure strategies. 

The second axiom that we consider says that a player is irrational if she 
chooses a particular strategy while believing that a different strategy is at 
least as good and she considers it possible that this alternative strategy is ac- 
tually better than the chosen one. We show that common knowledge of this 
stronger notion of rationality characterizes the iterated deletion procedure 
introduced by Stalnaker [17], restricted—once again—to pure strategies. 

The paper is organized as follows. In the next section we review the 
KD45 multi-agent logic for belief and common belief and the S5 logic for 
knowledge and common knowledge. In Section 3 we review the definition 


1 This characterization of rationalizability is true for two-player games and extends to 
n-player games only if correlated beliefs are allowed (see Brandenburger and Dekel [8]). 
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of strategic-form game with ordinal payoffs and the iterated deletion proce- 
dures mentioned above. In Section 4 we define game logics and introduce 
two axioms of rationality. In Section 5 we characterize common belief of 
rationality in the weaker sense and common knowledge of rationality in the 
stronger sense. The characterization results proved in Section 5 (Propo- 
sitions 5.4 and 5.8) are not characterizations in the sense in which this 
expression is used in modal logic, namely characterization of axioms in 
terms of classes of frames (see [3, p. 125]). Thus in Section 6 we provide 
a reformulation of our results in terms of frame characterization. In Sec- 
tion 7 we discuss related literature, while Section 8 contains a summary and 
concluding remarks. 


2 Multi-agent logics of belief and knowledge 


We consider a multi-modal logic with n + 1 operators B1, Bo,..., Bn, Bx 
where, for i = 1,...,n, the intended interpretation of By is “player i 
believes that vy”, while B,y is interpreted as “it is common belief that y”. 
The formal language is built in the usual way (see [3] and [10]) from a 
countable set A of atomic propositions, the connectives ~ and V (from 
which the connectives A, — and + are defined as usual) and the modal 
operators. 

We denote by KD457 the logic defined by the following axioms and 
rules of inference. 


Axioms: 


1. All propositional tautologies. 


2. Axiom K for every modal operator: for O € {Bi,..., Bn, Bs}, 


p aO > p) > Ov. (K) 


3. Axioms D, 4 and 5 for individual beliefs: for i=1,...,n, 


Bip > 7Bir¢, (D;) 
Bip > Bi Big, (4;) 

4. Axioms for common belief: for i =1,...,n, 
Buy > B; Buy, (CB2) 


B.(y > Biy A A Bay) > (Big A- A Bno > Buy). (CB3) 
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Rules of Inference: 


1. Modus Ponens: 


From ¢ and (p —> w) infer w. (MP) 
2. Necessitation for every modal operator: for O € {By,..., Bn, Bx}, 
From g infer Oy. (Nec) 


We denote by S5% the logic obtained by adding to KD45; the following 
axiom: 


5. Axiom T for individual beliefs: for i = 1,...,n, 
Bip > y. (T;) 


While KD45ž is a logic for individual and common beliefs, S5% is the 
logic for (individual and common) knowledge. To stress the difference be- 
tween the two, when we deal with S5% we shall denote the modal operators 
by K; and K, rather than B; and Bx, respectively. 

Note that the common belief operator does not inherit all the properties 
of the individual belief operators. In particular, the negative introspection 
axiom for common belief, ~B,y —> B,7B., is not a theorem of KD45%7. 
In order to obtain it as a theorem, one needs to strengthen the logic by 
adding the axiom that individuals are correct in their beliefs about what is 
commonly believed: B;B.p — B.. Indeed, the logic KD45ž augmented 
with the axiom B;B,y — By coincides with the logic KD45%ž augmented 
with the axiom =B, —> B,-B,¢ (see [6]). 


On the semantic side we consider Kripke structures (Q, B1,...,Bn, Bx) 
where Q is a set of states or possible worlds and, for every j € {1,...,n, *}, 
B; isa binary relation on Q.? For every w € Q and for every j € {1,...,n, *}, 


let Bj(w) = {w E Q : wBjw'}. 


Definition 2.1. A D45% frame is a Kripke structure (Q, Bi,...,Bn, Bx) 
that satisfies the following properties: for all w,w’ € Q andi=1,...,n 


1. Seriality: B;(w) 4 Ø; 
2. Transitivity: if w € B;(w) then B;(w’) C B;(w); 
3. Euclideanness: if w € B;(w) then B;(w) C B;(w’); 
Th rate bout tlie paper we sal use the Raman font tor Byntacliepemter le:s: B; 


and K;) and the Calligraphic font for the corresponding semantic relations (e.g., B; 
and K;). 
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B, : Chk (+) 


B Y 


2: 0 fos 
a B Y 
Be Oe 


FIGURE 1. Illustration of a D45% frame. 


4. B, is the transitive closure of B1 U---U Bn, that is, w € B,(w) if and 
only if there is a sequence (w1,...,Wm) in Q such that (1) wi = w, (2) 
Wm = w’ and (3) for every k = 1,...,m-—1 there is an ip E€ {1,...,n} 
such that Wk+1 €E Bi, (wk). 


An S5% frame is a D45% frame that satisfies the following additional 
property: for all w € Q and i = 1,..., Nn, 


5. Reflexivity: w € B;(w). 


Figure 1 illustrates the following D45% frame: n = 2, Q = {a, 68,7}, 
Bı(a) = Bi(8) = {a}, Bily) = {7}, B2(a) = {a} and B2(6) = B2(7) = 
{8,7}. Thus B.(a) = {a} and B,(3) = B.(y) = {a, 8, y}. We shall use 
the following convention when representing frames graphically: states are 
represented by points and for every two states w and w’ and for every 
j € {1,...,n,*}, w € B;(w) if and only if either (i) w and w’ are enclosed 
in the same cell (denoted by a rounded rectangle), or (ii) there is an arrow 
from w to the cell containing w’, or (iii) there is an arrow from the cell 
containing w to the cell containing w’. 

The link between syntax and semantics is given by the notions of valu- 
ation and model. A D45% model (respectively, S5% model) is obtained by 
adding to a D45* frame (respectively, S5% frame) a valuation V : A > 2®, 
where A is the set of atomic propositions and 2° denotes the set of subsets 
of Q. Thus a valuation assigns to every atomic proposition p the set of 
states where p is true. Given a model and a formula y, we denote by w = y 
the fact that ọ is true at state w. The truth set of y is denoted by |||, 
that is, ||| = {w € Q : w = y}. Truth of a formula at a state is defined 
recursively as follows: 
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ifp €A, w = p if and only if w € V (p), 

w = ng if and only if w É ọ, 

wH vy if and only if either w = y or w } ẹ (or both), 

w H Bip if and only if B;(w) C ||y||, that is, 

(j=1,...,n) if w H ¢ for all w € B;(w), 

w H Bx if and only if B,(w) C |lyl]. 

A formula y is valid in a model if it is true at every state, that is, if 
|y|| = Q. It is valid in a frame if it is valid in every model based on that 
frame. 


The following result is well-known:? 


Proposition 2.2. The logic KD45* is sound and complete with respect 
to the class of D45* frames, that is, a formula is a theorem of KD45% if 
and only if it is valid in every D45% frame. Similarly, S57, is sound and 
complete with respect to the class of S5% frames. 


3 Ordinal games and dominance 


In this paper we restrict attention to finite strategic-form (or normal-form) 
games with ordinal payoffs, which are defined as follows. 


Definition 3.1. A finite strategic-form game with ordinal payoffs is a quin- 
tuple G = (N, {Si hien, O, {=i ien, 2), where 


e N ={1,...,n} is a set of players, 
e S; is a finite set of strategies of player i € N, 
e © is a finite set of outcomes, 


>; is player i’s ordering of O,* 


z : S — O (where S = S1 x---x Sn) is a function that associates with 
every strategy profile s = (s1,..., Sn) an outcome z(s) € O. 


Given a player i we denote by S_; the set of strategy profiles of the 
players other than i, that is, S_; = S1 X ++- x Si—1 X Sj41 X- X Sn. When 
we want to focus on player i we shall denote the strategy profile s € S by 
(si, s—i) where s; € S; and s_; € S-i. 


3 See [4]. The same result has been provided for somewhat different axiomatizations of 
common belief by a number of authors (for example [14], [15] and [12]). 

4 That is, +; is a binary relation on O that satisfies the following properties: for all 
o, 0’, 0’ € O, (1) either o >; o' or o' =; o (completeness or connectedness) and (2) 
if o >; o' and o/ >; o” then o >; o” (transitivity). The interpretation of o =; o’ is 
that, according to player i, outcome o is at least as good as outcome o’. The strict 
ordering >; is defined as usual: o >; o’ if and only if o >; o’ and not o’ >; o. The 
interpretation of o >; o’ is that player i strictly prefers outcome o to outcome o’. 
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FIGURE 2. Illustration of the iterated deletion of strictly dominated strategies. 
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Definition 3.2. Given a game G = (N,{Si}ien,O,{=i}ien, 2) and s; E 
Si, we say that, for player i, s; is strictly dominated in G if there is an- 
other strategy t; E€ S; of player 7 such that—no matter what strategies the 
other players choose—player i prefers the outcome associated with t; to 
the outcome associated with s;, that is, if z(t;,s_;) >; 2(s:,s—:), for all 
si © Si. 


Let G = (N, {Si}ien, O, {zi }ien, z) and G’ be two games, where G” = 
(N', {Sihen O', {= }ien, z). We say that G’ is a subgame of G if N’ = 
N, O' = O, for every i € N: >; = >; and Si C S; (so that S’ C S) 
and z’ coincides with the restriction of z to S’ (that is, for every s’ € S”, 


z'(s') = 2(s')). 


Definition 3.3 (IDSDS procedure). The Iterated Deletion of Strictly Dom- 
inated Strategies is the following procedure. Given a game G = 
(N, {Si}ien, O, {=ihien, z) let (G°,G',...,G™,...) be the sequence of sub- 
games of G defined recursively as follows. For all i € N, 


1. let S? = S; and let D? C S? be the set of strategies of player i that 
are strictly dominated in G? = G; 


2. for m > 1, let S™ = S~1\D'"! and let G™ be the subgame of G 
with strategy sets S7”. Let D} C S™ be the set of strategies of player 
i that are strictly dominated in G”. 


Let S?° = (men 9?” (where N denotes the set of non-negative integers) 
and let G be the subgame of G with strategy sets S°. Let S% = SP x 
Sal 5995 


The IDSDS procedure is illustrated in Figure 2, where: 


S?={A,B,C,D} Di ={D} Sy={e,f,g} D= 
Si ={4,B,C} Di=2, S3={e,f,g} Dr={9} 


Si = {A, B,C} Di ={C} Sz = {e, f} D; =ø 
SÌ = {A, B} Dï=Øø S3= {e, f} D3 = {f} 
St = {A, B} Di={B} S% =59}={e} D}=2 
S? = S} = {A} 


Thus S% = {(A,e)}. 
In Figure 2 we have represented the ranking >; by a utility (or payoff) 
function w; : S — R satisfying the following property: u;(s) > u;(s’) if and 


5 Note that, since the strategy sets are finite, there exists an integer r such that G® = 
GT = GTt* for every k EN. 
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only if z(s) =; z(s’) (in each cell, the first number is the payoff of player 1 
while the second number is the payoff of player 2).° 

The next iterated deletion procedure differs from IDSDS in that at every 
round we delete strategy profiles rather than individual strategies. This 
procedure is the restriction to pure strategies of the algorithm introduced 
by Stalnaker [17]. 


Definition 3.4 (IDIP procedure). Let G = (N, { Sihen, O, {=i }ien, 2), be 
a game, together with a subset of strategy profiles X C S and a strategy 
profile x € X. We say that x is inferior relative to X if there exists a player 
i and a strategy s; € S; of player i (thus s; need not belong to the projection 
of X onto S;) such that: 


1. z(si,£—i) >; 2(@i, 2-1), and 
2. for all s_; E€ S-i, if (aj, s_;) E€ X then 2(s;, s_;) =; Z(£i, S—i). 


The Iterated Deletion of Inferior Profiles (IDIP) is defined as follows. For 
m € N define T™ C S recursively as follows: T° = S and, for m > 1, 
T™ =T™1\I™"!, where I™-! C T™~! is the set of strategy profiles that 


are inferior relative to T”—*. Let T° = Nmen T” 


The IDIP procedure is illustrated in Figure 3, where 


S=T°= {(4,d), (Ave), (A, f), (B,d), (B,e), (B, f), (C, d), (C, e), (C, f)}, 
P= {(B, e), (C, f)} 


(the elimination of (B, e) is done through player 2 and strategy f, while the 
elimination of (C, f) is done through player 1 and strategy B); 


T’ = {(A, d), (A, e), (A, f), (B, d), (B, f), (C, d), (C, e)}, 
r= {(B, d), (B, f), (C,e)} 


(the elimination of (B,d) and (B, f) is done through player 1 and strategy 
A, while the elimination of (C,e) is done through player 2 and strategy d); 


T? = aA d), (A, e), (A, f), (C, d)}, 
P ={(C,d)} 


6 Note that the payoff function u; : S — R used in Figure 2 to represent the ranking >; 
of player i is an ordinal function in the sense that it could be replaced by any other 
function v; obtained by composing u; with a strictly increasing function on the reals. 
That is, if f; : R — R is such that f(x) > fi(y) whenever x > y, then v; : S > R 
defined by v;(s) = fi(ui(s)) could be used as an alternative representation of >; and 
the outcome of the IDSDS algorithm would be the same. 

T Since the strategy sets are finite, there exists an integer r such that T® = T” = T7+* 
for every k € N. 
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FIGURE 3. Illustration of the iterated deletion of inferior strategy profiles. 


(the elimination of (C, d) is done through player 1 and strategy A); 


T? > {(A, d), (A, e), (A, P 
P=øØ 


bi 


and thus T® = T. 


4 Game logics 


A logic is called a game logic if the set of atomic propositions upon which 
it is built contains atomic propositions of the following form: 


e Strategy symbols s;, ti, ... The intended interpretation of s; is “player 
i chooses strategy s;”. 


e The symbols r; whose intended interpretation is “player 7 is rational”. 


e Atomic propositions of the form t; =; si, whose intended interpreta- 
tion is “strategy t; of player 7 is at least as good, for player 7, as his 
strategy s;”, and atomic propositions of the form t; >; si, whose in- 
tended interpretation is “for player i strategy t; is better than strategy 


” 
Si . 


From now on we shall restrict attention to game logics. 
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Definition 4.1. Fix a game G = (N, {Sihen O, {=i hien, z) with S; = 
{s}, s?,..., s7} (thus the cardinality of S; is m;). A game logic is called 


re ae 


jaie 


abuse of notation we use the symbol s* to denote both an element of Sj, that 
is, a strategy of player i, and an element of A, that is, an atomic proposition 
whose intended interpretation is “player i chooses strategy s*”). 


Given a game G with S; = {s/,s?,..., si}, we denote by L2* (re- 
spectively, LŽ) the KD45* (respectively, S5*) G-logic that satisfies the 
following additional axioms: for all i =1,...,n and for all k,€=1,...,mi, 
with k Æ 4, 


(sł V s? V- V a) (G1) 
(8? A s4), (G2) 

s* > Bis}, (G3) 
(G4) 

(G5) 


(sf =i si) V (sf =: sf), G4 
G5 


i 


(si =i sf) > ((5 =: 87) A (Ei 89). 


Axiom G1 says that player i chooses at least one strategy, while axiom G2 
says that player i cannot choose more than one strategy. Thus G1 and 
G2 together imply that each player chooses exactly one strategy. Axiom 
G3, on the other hand, says that player i is aware of his own choice: if 
he chooses strategy s* then he believes that he chooses s*?. The remaining 
axioms state that the ordering of strategies is complete (G4) and that the 
corresponding strict ordering is defined as usual (G5). 


Proposition 4.2. Fix an arbitrary game G. The following is a theorem of 
logic LB: B;s¥ — s}. That is, every player has correct beliefs about her 
own choice of strategy.® 


Proof. In the following PL stands for Propositional Logic. Fix a player i 
and k,l € {1,...,m,;} with k 4 £. Let p denote the formula 


(sV e V SE) Asi A--- Ans?! ASETI A... Aas, 


1 pos tautology 
2. Slsk A s£) axiom G2 (for £ Æ k) 
3. sk = sé 2, PL 


8 Note that, in general, logic LR allows for incorrect beliefs. In particular, a player 
might have incorrect beliefs about the choices made by other players. By Proposition 
4.2, however, a player cannot have mistaken beliefs about her own choice. 
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4 Bist = Bins! 3, rule RK? 

5. Bast —. aB;s* axiom D; 

6. si — Bist axiom G3 

7 AB; s< > sé 6, PL 

8. Bis 4 ~ns! 4,5, 7, PL (for £ Æ k) 

9. sVe V si axiom G1 
10. Bysk + (stV---V si") 9, PL 
11. Bist sy 8 (for every £ Æ k), 10, PL 
12. Bis — s} 1, 11, PL 


Q.E.D. 


On the semantic side we consider models of games, which are defined as 
follows. 


Definition 4.3. Given a game G = (N,{Si}ien,O, {z;i}ien, z) and a 
Kripke frame F = (0, {Bi}ien, Bz), a frame for G, or G-frame, is ob- 
tained by adding to F n functions c; : Q — S; (i € N) satisfying the 
following property: if w € B;(w) then o;(w’) = a;(w). 


Thus a G-frame adds to a Kripke frame a function that associates with 
every state w a strategy profile o(w) = (o1(w), ...,on(w)) € S. The restric- 
tion that if w” € B;(w) then o;(w’) = o;(w) is the semantic counterpart to ax- 
iom G3. Given a player i, as before we will denote o(w) by (a;(w), a_-i(w)), 
where o_;(w) € S_; is the profile of strategies of the players other than i. 

We say that the G-frame (Q, {Bi}ien, Bx, {oi}ien}) is a D45% G-frame 
(respectively, S5% G-frame) if the underlying Kripke frame (Q, {Bi}ien, Bx) 
is a D45* frame (respectively, S5% frame: see Definition 2.1). 


Fe = (Q, {Bitien, Bx, {oi }ien}), a model of G, or G-model, is obtained by 
adding to Fg the following valuation: 


Definition 4.4. Given a game G with S; = {s},s?,...,s/""}, anda G-frame 


e w H sè? if and only if o;(w) = s}, 


e w H (s£ >; s£) if and only if z(s¥, 0_;(w)) =; (sf, o_i(w)). 


Thus at state w in a G-model it is true that player i chooses strategy s? 
if and only if the strategy of player i associated with w is s? (a;(w) = s?) 
and it is true that strategy s¥ is at least as good as strategy s£ if and only 
if s* in combination with o_;(w) (the profile of strategies of players other 
than i associated with w) yields an outcome which player i considers at least 


as good as the outcome yielded by s£ in combination with o_;(w). 


9 RK denotes the inference rule “from Y —> x infer Oy > Oy ”, which is a derived rule 
of inference that applies to every modal operator LJ that satisfies axiom K and the 
rule of Necessitation. 
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al 2,1 Ol 24 
b}10 10 1,1 
c|1,4 13 0,3 


FIGURE 4. A game: player 1 controls the rows (i.e., has strategies a, b 
and c), and player 2 the columns. 


a B y 
a O Te 
a B Y 


FIGURE 5. D45% frame for the game of Figure 4. 


Let F2* (respectively, F%?) denote the set of D45% (respectively, S5% ) 
G-frames and M2*° (respectively, M@?) the corresponding set of G-models. 

Figure 4 illustrates a two-player game with strategy sets Sı = {a,b,c} 
and S2 = {d,e, f} and Figure 5 a D45% frame for it. The corresponding 
model is given by the following valuation: 


aH bren (b> a) A (c> a) A (ba e)la b) 
A^ (f =2 d) A (f =2 e) A (e =2 d) A (d =2 e), 
BEbAdA (a> b)A (a > c) A (ba c) A (ea b) 
A (f =2 d) A (f =2 e) A (e =2 d) A (d =2 e), 
( 
\( 


yE add (a>; b) A (a> c) A (ba Cc) A (ea 6) A (d &2 e) 
erg d) A (d =2 f) A (F =2 d) A (e Ha f) A (F = €). 


Proposition 4.5. Logic LE^ (respectively, L$#) is sound with respect to 
the class of M2*° (respectively, M??) models. 
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Proof. It follows from Proposition 2.2 and the following observations: (1) 
axioms G1 and G2 are valid in every model because, for every state w there 
is a unique strategy s* € S; such that o;(w) = së and, by the validation 
rules (see Definition 4.4), w = s% if and only if o;(w) = s*; (2) axiom G3 is 
an immediate consequence of the fact (see Definition 4.3) that if w € B;(w) 
then o;(w’) = o;(w); (3) axioms G4 and G5 are valid because, for every 
state w, there is a unique profile of strategies o_;(w) of the players other 
than i and the ordering >; on O restricted to z(S; x o_;(w)) induces an 
ordering of $;. Q.E.D. 


5 Rationality and common belief of rationality 


So far we have not specified what it means for a player to be rational. 
The first extension of LRS that we consider captures a very weak notion 
of rationality. The following axiom—called WR for ‘Weak Rationality’ — 
says that a player is irrational if she chooses a particular strategy while 
believing that a different strategy is better for her (recall that r; is an 
atomic proposition whose intended interpretation is “player 7 is rational” ): 

sE A Bils =; sE) > ary. (WR) 

Given a game G, let L2*°+WR (respectively, L2?+WR) be the exten- 
sion of L2*° (respectively, L3) obtained by adding axiom WR to it. 

The next axiom—called SR for ‘Strong Rationality —expresses a slightly 
stronger notion of rationality: it says that a player is irrational if she chooses 
a strategy while believing that a different strategy is at least as good and 
she considers it possible that this alternative strategy is actually better than 
the chosen one: 

sE A Bi(s’ >; sE) AABi(sf =; sk) > ary. (SR) 

Given a game G, let L@*°+SR (respectively, L??+SR) be the extension 

of LB45 (respectively, L@?) obtained by adding axiom SR to it. 


The following shows that L2*°+SR is an extension of L2*°+WR. 
Proposition 5.1. WR is a theorem of L2*+SR. 


Proof. As before, PL stands for Propositional Logic. 


1. sk A Bi(sf =i sf) AABi7(sf > sf) > ori Axiom SR 
2. (ri As) > 7(Bi(sf =; sE) A ABi7(sf >; s¥)) 1, PL 

3. (sf =i s¥) => (s£ =i sE) A(s} >; s£) Axiom G5 
4. (sf =i sE) > (sf =i s$) 3, PL 

5. Bi(sf >; sf) > Bi(sf >; sf) 4, RK 

6. Bi(st =i st) a =B;=(s£ >i s£) Axiom D; 
T. B;(s{ =i s¥) > (Bi sf an} sk) A =~B;=(sf >i ery) 5, 6, PL 
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8. -(Bi(sf =; s£) A AB;7(sf >; s*)) > AB; (sf >; s£) 7, PL 


a 


9. (ri A s*) 3 AB;(s' >; s£) 2, 8, PL 


t 


10. sk A Bi(sf >; sE) > ar; 9, PL 


t 
Q.E.D. 
ote s D45|WR S5|WR s 
Definition 5.2. Given a game G, let Mg C MR” (Me c M2) 


be the class of D45% (respectively, S5% ) G-models (see Definition 4.4) where 
the valuation function satisfies the following additional condition: 


e w = r; if and only if, for every s; € S; there exists an w’ € B;(w) such 
that z(oilw), o_i(w’)) =i z(si, o—ilw")).t2 


Thus at state w player i is rational if and only if, for every strategy 
si of hers, there is a state w’ that she considers possible at w (w’ € B;(w)) 
where the strategy that she actually uses at w (a;(w)) is at least as good as s; 
against the strategies used by the other players at w’ (o_;(w’)). For instance, 
in the model based on the frame of Figure 5 we have that a = (rı A 719), 
BE (rı A r2) and y = (rı A r2). To see, for example, that G = r2 note 
that o2(3) = d and for strategy f we have that y € B2(8), o1(y) = a and 
z(a,d) =2 z(a, f), while for strategy e we have that 8 € Bo(8), 01(8) = b 
and z(b, d) =2 z(b,e). Thus, in the model based on the frame of Figure 5, we 
have that at state 8 both players are rational, player 2 believes that player 
1 is rational, but player 1 mistakenly believes that player 2 is irrational: 
B = ri A T2 A Bory A Bioro. 


Proposition 5.3. Logic L2+WR (respectively, L -+WR) is sound with 
respect to the class of models MR IWR (respectively, Me lWR), 

Proof. By Proposition 4.5 it is sufficient to show that axiom WR is valid in 
an arbitrary such model. Suppose that w E s¥\B;(sf >; s£). Then o;(w) = 
së and B;(w) C ||sf >; s*||, that is (see Definition 4.4), z(sf,a_;(w’)) >; 
2(s¥,o_i(w’)), for every w € B;(w). It follows from Definition 5.2 that 
w = ar;. Q.E.D. 


The following proposition says that common belief of the weak notion 
of rationality expressed by axiom WR characterizes the Iterated Deletion 
of Strictly Dominated Strategies (see Definition 3.3).1! 


10 This could alternatively be written as z(a;(w’),o_i(w’)) =; 2(si,7—i(w’)), since, by 
definition of G-frame (see Definition 4.3), if w’ € Bi(w) then o;(w’) = o;(w). 

11 Proposition 5.4 is the syntactic-based, ordinal version of a semantic, probabilistic- 
based result of Stalnaker [17]. As noted in the Introduction, Stalnaker’s result was, 
in turn, a reformulation of earlier results due to Bernheim [2], Pearce [16], Tan and 
Werlang [18] and Brandenburger and Dekel [8]. 

The characterization results given in Propositions 5.4 and 5.8 are not characteriza- 
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Proposition 5.4. Fix a finite strategic-form game with ordinal payoffs G. 
Then both (A) and (B) below hold. 


(A) Fix an arbitrary model in i) oa and an arbitrary state w. 
If w H Ba(r1 A+++ Arn) then o(w) € 85%. 
(B) For every s € S% there exists a model in 1 and a state w such 


that (1) o(w) = s and (2) w = Ky(ri A+++ A Tn)! 


Proof. (A) Fix a model in Me and a state a and suppose that a } 
B,(r1 ^+- A^ Tn). The proof is by induction. First we show that, for every 
player i = 1,...,n and for every w € B,(a), o:(w) ¢ D} (see Definition 
3.3). Suppose not. Then there exist a player i and a 8 € B,(a) such that 
oi(3) € DÌ, that is, strategy o;(@) of player i is strictly dominated in G by 
some other strategy 8; € Si: for every s_; E S_;, 2(8;,5-i) >i z(oi(G), $i). 
Then, for every w € B;(G), 2(8:, 7-i(w)) >: 2(0i(8), o—(w)). It follows from 
Definition 5.2 that 8 = 7r;, contradicting the hypothesis that 8 € B,(a) 
and a — B,r;. Since, for every w € Q, oi(w) € S} = Sj, it follows that, 
for every w € B,(a), ai(w) € SP\D? = S}. Next we prove the inductive 
step. Fix an integer m > 1 and suppose that, for every player j = 1,...,n 
and for every w € B.(a), oj(w) € 87°. We want to show that, for every 
player i = 1,...,n and for every w € B,(a), o;(w) ¢ D}. Suppose not. 
Then there exist a player i and a 8 € B,(a) such that o;(8) € D7”, that is, 
strategy o;(() is strictly dominated in G™ by some other strategy 5; € S?”. 
Since, by hypothesis, for every player j and for every w € B,(a), oj;(w) € 
S™, it follows—since B;(3) C B,(3) C B.(a) (see Definition 2.1)—that for 
every w E€ B,(G), 2(8;,7_i(w)) >: z(o:(8), o-i(w)). Thus, by Definition 5.2, 
B = ~ri, contradicting the fact that 6 € B.(a) and a = B,r;. Thus, for 
every player i = 1,...,n and for every w € B,(a), oilw) E€ men S = SP. 
It only remains to show that o;(a) € S9°. Fix an arbitrary 6 € B;(a). 
Since B;(a) C B.(a), 8 € B.(a). Thus o;(8) € S. By Definition 4.3, 
oilb) = cila). Thus c;la) € SP9. 

(B) Let m be the cardinality of S% = SP x- x S and let Q = 
{w1,..., Wm}. Let o : Q > S” be a one-to-one function. For every player i, 


tions in the sense in which this expression is used in modal logic, namely characteri- 
zation of axioms in terms of classes of frames (see [3, p. 125]). In Section 6 we provide 
a reformulation of Propositions 5.4 and 5.8 in terms of frame characterization. 

Recall that, in order to emphasize the distinction between belief and knowledge, when 
dealing with the latter we denote the modal operators by K; and K, rather than B; 
and Bą, respectively. Similarly, we shall denote the accessibility relations by K; and 
K, rather than 5; and Bx, respectively. 

Thus while part (A) says that if at a state there is common belief of rationality then 
the strategy profile played at that state belongs to the set of strategy profiles that are 
obtained by applying the IDSDS algorithm, part (B) says that any such strategy profile 
is realized at a state of some model where there is common knowledge of rationality 
(that is, common belief with the added property that individual beliefs satisfy the 
Truth Axiom T;). 


12 
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define the following equivalence relation on Q: wK jw’ if and only if o;(w) = 
a;(w’), where o;(w) is the ith coordinate of o(w). Let K, be the transitive 
closure of [J;e y Ki (then, for every w € Q, K,(w) = Q). The structure so 
defined is clearly an S5% G-frame. Consider the model corresponding to this 
frame (see Definition 4.4). Fix an arbitrary state w and an arbitrary player 
i. By definition of S%, for every s; € S; there exists an w’ € K;(w) such 
that z(o;(w),0_;(w’)) zi 2(si,0_;(w’)). Thus w H r; (see Definition 5.2). 
Hence, for every w € Q, w H (rı A++- Arn) and, therefore, for every w € Q, 
w H Klr A+++ Arn). Q.E.D. 


Remark 5.5. Since MWR Cc Mg IWR it follows from part (B) of Propo- 
sition 5.4 that the implications of common belief of rationality—as implic- 
itly defined by axiom WR—are the same as the implications of common 
knowledge of rationality. 


The above observation is not true for the stronger notion of rationality 
expressed by axiom SR, to which we now turn. 


Definition 5.6. Given a game G, let MBI? c MB (MẸ c MS, 
respectively) be the class of D45 (respectively, S5) G-models where the 
valuation function satisfies the following condition: 


e w = r; if and only if, for every s; € Si, whenever there exists an 
w’ € B;(w) such that 2(s;,0_;(w’)) >: z(oilw),o—ilw')) then there 
exists an w” € B;(w) such that z(o;(w), 0_;(w”)) >: 2(si,7_;(w”)). 


Thus, at state w, player i is rational if, whenever there is a strategy s; 
of hers which is better than o;(w) (the strategy she is actually using at w) 
at some state w’ that she considers possible at w, then o;(w) is better than 
si at some other state w” that she considers possible at w. For example, in 
the model based on the frame of Figure 5 we have that w = (r1 A =r2) for 
every w E {a, 8, y}. At state 8, for instance, player 2 is choosing strategy 
d when there is another strategy of hers, namely f, which is better than 
d at B and as good as d at y, and B2(8) = {3,7}. Thus she is irrational 
according to Definition 5.6. 

It is easily verified that My, Isk C MRSIWR 


S5|SR S5|WR 
that Me? c MIR, 


and, similarly, it is the case 


Proposition 5.7. Logic L2*°+SR (respectively, L -+SR.) is sound with 


respect to the class of models Mas (respectively, Me SR), 


Proof. By Proposition 4.5 it is sufficient to show that axiom SR is valid in an 
arbitrary such model. Suppose that w = s* A B;(sf =; s*) A7B;7(s' >; s*). 
Then o;(w) = s* and B;(w) C ||sf =; s*|| [that is—see Definition 4.4— 
2(sf,o0_;(w’)) =; 2(s¥,o_;(w’)), for every w € B;(w)] and there is an w” € 
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B;(w) such that w” — sf >; s*, that is, 2(sf,o_i(w")) >; z(s¥,o_;(w”)). It 


4? 


follows from Definition 5.6 that w = 717;. Q.E.D. 


The following proposition says that common knowledge of the stronger 
notion of rationality expressed by axiom SR characterizes the Iterated Dele- 
tion of Inferior Profiles (see Definition 3.4).1° 


Proposition 5.8. Fix a finite strategic-form game with ordinal payoffs G. 


Then both (A) and (B) below hold. 


A) Fix an arbitrary model in Meese and an arbitrary state w. If 
y G y 


w H K4(rı A+ Arn) then o(w) E T”. 
(B) For every s € T® there exists a model in Me Sk and a state w such 
that (1) o(w) = s and (2) w H K,(r1 A+++ Arn). 


Proof. (A) As in the case of Proposition 5.4, the proof is by induction. Fix a 
model in MS and a state a and suppose that a H K.(riA---Ary). First 
we show that, for every w € K.(a), o(w) ¢ I? (see Definition 3.4). Suppose, 
by contradiction, that there exists a 8 € K.(a) such that o(@) € T°, that 
is, o(@) is inferior relative to the entire set of strategy profiles S. Then 
there exists a player i and a strategy 8; € S; such that 2(8;,0_;(8)) >: 
z(0;(8),o0—<(8)), and, for every s_; E€ S-i, 2(8;, 8-1) =; z(0;(G), s_;). Thus 
2(8;,0-i(w)) =; z(oi(G),7_i(w)), for every w € Kil); furthermore, by 
reflexivity of K; (see Definition 2.1), 6 € K;(3). It follows from Definition 
5.6 that 6 = ~ri. Since 6 € K,(a), this contradicts the hypothesis that 
a — K,r;. Thus, since, for every w € Q, o(w) € S = T° we have shown 
that, for every w € K,(a), olw) € TOW = T}. 

Now we prove the inductive step. Fix an integer m > 1 and suppose 
that, for every w € K,(a), o(w) € T™. We want to show that, for ev- 
ery w € K,(a), o(w) € I™. Suppose, by contradiction, that there ex- 
ists a 8 € K.(a) such that o(86) € I™, that is, ø(8) is inferior relative 
to T™. Then there exists a player 7 and a strategy 5; € S; such that 
z(5i,o—:il(ß)) >: z(0i(B), o—:(8)), and, for every s_; € S_i, if (S;,5_;) E T™ 
then z(5;,8-;) =; z(oi(G),s-:). By Definition 4.3, for every w € K;(), 
o;(w) = o,(8) and by the induction hypothesis, for every w € K,(a), 
(ai(w),o_i(w)) € T™. Thus, since K;(8) C K,(8) C K.(a), we have 
that, for every w € K,(G), (oi(G),c_i(w)) E€ T™. By reflexivity of Ki, 
B €K;(G). It follows from Definition 5.6 that 8 = ~ri. Since 8 € K,(a), 
this contradicts the hypothesis that a = K,r;. 

Thus, we have shown by induction that, for every w € K.(a), olw) € 
Omen T = T°. It only remains to establish that o(a) € T°, but this 
follows from reflexivity of Kx. 


13 Proposition 5.8 is the syntactic-based, ordinal version of a semantic, probabilistic- 
based result due to Stalnaker [17]. For a correction of that result see Bonanno and 
Nehring [5]. 
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FIGURE 6. A game where player 1 has strategies a and b, and player 2 has 
c and d. 


(B) Let m be the cardinality of T° and let Q = {w1,..., Wm}. Let 
o : Q — T” be a one-to-one function. For every player i, define the 
following equivalence relation on Q: ww’ if and only if o;(w) = oilw’), 
where g;(w) is the ith coordinate of o(w). Let K., be the transitive closure 
of Uien Ki (then, for every w € 0, K,(w) = Q). The structure so defined 
is clearly an S5% G-frame. Consider the model corresponding to this frame 
(see Definition 4.4). Fix an arbitrary state w and an arbitrary player i. 
By definition of T°, for every player i and every s; € S; if there exists 
an w € K;(w) such that if z(s;,0_;(w’)) >; z(o;(w),o_;(w’)) then there 
exists an w” € K;(w) such that z(o;(w),o_i(w”)) >; 2(s;,0-;(w”)). Thus 
w = r; (see Definition 5.6). Hence, for every w E€ Q, w H (r1 A+++ ^rn) and, 
therefore, for every w E€ Q, w H Ka(r1 A+++ Arn). Q.E.D. 


Note that Proposition 5.8 is not true if one replaces knowledge with be- 
lief, as illustrated in the game of Figure 6 and corresponding frame in Fig- 
ure 7. In the corresponding model we have that, according to the stronger 
notion of rationality expressed by Definition 5.6, a = r1 Arg and 8 = r1 ^ra, 
so that a — B,(r1 A r2), despite the fact that o(a) = (b, d), which is an 
inferior strategy profile (relative to the entire game).14 In other words, com- 
mon belief of rationality, as expressed by axiom SR, is compatible with the 
players collectively choosing an inferior strategy profile. Thus, unlike the 
weaker notion expressed by axiom WR (see Remark 5.5), with axiom SR 
there is a crucial difference between the implications of common belief of 
rationality and those of common knowledge of rationality. 


6 Frame characterization 


The characterization results proved in the previous section (Propositions 
5.4 and 5.8) are not characterizations in the sense in which this expression 
is used in modal logic, namely characterization of axioms in terms of classes 
of frames (see [3, p. 125]). In this section we provide a reformulation of our 
results in terms of frame characterizations. 


Definition 6.1. An axiom characterizes (or is characterized by) a class F 


14 In the game of Figure 6 we have that S® = S = {(a,c), (a, d), (b,c), (b,d)} while 
T™® = {(a,c), (b, c)}. 
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FIGURE 7. A frame for the game of Figure 6. 


of Kripke frames if the axiom is valid in every model based on a frame that 
belongs to F and, conversely, if a frame does not belong to F then there is a 
model based on that frame and a state in that model at which an instance 
of the axiom is falsified.15 


We now modify the previous analysis as follows. First of all, we drop 
the symbols r; from the set of atomic propositions and correspondingly 


drop the definitions of the classes of models Mm vee Ma 


and Mase (Definitions 5.2 and 5.6). Secondly we modify axioms WR and 
SR as follows: 


sf > =B;(sf >i sf), (WR’) 
s} > 7(Bi(s) =i sf) A7Bi-(s >i s¥)). (SR’) 


One can derive axioms WR’ and SR’ from the logics considered previ- 
ously by adding the axiom that players are rational. In fact, from r; and 
WR one obtains WR’ (using Modus Ponens) and similarly for SR’. 


The next proposition is the counterpart of Proposition 5.4. 


Proposition 6.2. Subject to the valuation rules specified in Definition 4.4 
for the atomic propositions a and (sé am sÉ), axiom WR’ is character- 
ized by the class of D45% game frames (see Definition 4.3) that satisfy the 
following property: for all i € N and for all w € Q, o;(w) € SP. 


Proof. Fix a model based on a frame in this class, a state a, a player i 
and two strategies s* and s! of player i. Suppose that a } s*, that is, 


15 For example, as is well known, the axiom Bip — B;B;y is characterized by the class 
of frames where the relation 6; is transitive. 
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cila) = s*. We want to show that a H ~B;(s‘ >; s£). Suppose not. Then 
B(a) C ||sf =; s*l|, that is, 


for every w € Bila), z(sf,0_;(w)) >; 2(s*,0_;(w)). (6.1) 


By hypothesis, for every player j # i and for every w € Q, oj;(w) € SP. 
Thus it follows from this and (6.1) that s* ¢ $°°, contradicting the hy- 
potheses that o;(a@) = së and o;(w) € SP for allw EQ. 

Conversely, fix a D45% frame not in the class, that is, there is a state 
w E€ Q and a player i € N such that o;(w) ¢ S9°. For every state w and 
every player j let 


ewe oo) 1 o;(w) € om 
m if oj;(w) E€ DP. 
Let m = min{m(w, j): j E N,w E€ Q}. By our hypothesis about the frame, 
mM EN. Let i € N anda E Q be such that rh = m(a,i). Then 


ai(a) € DË (6.2) 


and, since (see Definition 3.3), for every j € N and for every p,q E NU{00}, 
be Gee 
jo Sja 
for every j E€ N andw EQ, oj(w) € 87. (6.3) 


Let s* = o;(a). By (6.2) and (6.3) , there exists a sf € S; such that, for 
everyw EQ, 2(sf,0_i(w)) >: 2(s¥, 7_i(w)). Thus B;(a) C ||sf >; s*|| and 
thus a H s* A B;(sf +; s*), so that axiom WR’ is falsified at a. Q.E.D. 


t 


The next proposition is the counterpart of Proposition 5.8. 


Proposition 6.3. Subject to the valuation rules specified in Definition 4.4 
for the atomic propositions s? and (sf >; s¥), axiom SR’ is characterized by 
the class of S5% game frames (see Definition 4.3) that satisfy the following 
property: for all w € Q, o(w) E T”. 


Proof. Fix a model based on a frame in this class, a state a, a player 7 and 
two strategies s: and s£ of player i. Suppose that a = s¥ A Kj(sf =; s*). 
Then o;(a) = s¥ and Kila) C ||sf =; s*||, that is, 


t 


for all w € Kila), z(s£,o—ilw)) =: z(sE,o-i(w)). (6.4) 


We want to show that a = K;7(sf +; s£). Suppose not. Then there exists 


a 3 € Kila) such that 8 H (sf >; s*), that is, 


a 


2(8;,0-i()) =i 2(87', 0-«(9)). (6.5) 
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It follows from (6.4) and (6.5) that (s£, 7_;(8)) = (0: (8), o—:(8)) is inferior 
relative to the set {s € S: s = o (w) for some w € K;(a)}, contradicting the 
hypothesis that o(w) € T™ for all w € Q. 
Conversely, fix an S5* frame not in the class, that is, there is a state 

w E Q such that o(w) ¢ T°. For every w € Q, let 

oo ifa(w) ET, 

m ifo(w) € LST! perk 
Let mo = min{m(w) : w € Q}. By our hypothesis about the frame, mo € N. 


Let a € Q be such that mo = m(a). Then o(a) € I™, that is, there is a 
player i and a strategy s£ € S; such that 


z(s£, a—i(a)) >; z(cila),o—ila)) (6.6) 


and 


Vw EQ, if (o;(a),o_i(w)) E T™ 
then z(sf,o0_;(w)) >; z(o;(a),0_;(w)). (6.7) 


By definition of mo, since (see Definition 3.4) for every p,q € NU {oo}, 
TP+4 C TP, for every w € Q, o(w) € T™. Thus, letting së = o;(a), it 
follows from (6.7) that K;(a) C ||sf =; s*||, that is, a H Ki(sf =; s*). Since 


the frame is an S5 frame, K; is reflexive and, therefore, a € K;(a). It follows 
from this and (6.6) that a H} ~K;7(s{ >; s¥). Thus a H së A K;i(sf >; 


a 


sk) A AK;7(s£ =; s£), so that axiom SR’ is falsified at a. Q.E.D. 


a 


There appears to be an important difference between the results of Sec- 
tion 5 and those of this section, namely that, while Propositions 5.4 and 
5.8 give a local result, Propositions 6.2 and 6.3 provide a global one. For 
example, Proposition 5.4 says that if at a state there is common belief of ra- 
tionality, then the strategy profile played at that state belongs to S°°, while 
its counterpart in this section, namely Proposition 6.2, says that the strategy 
profile played at every state belongs to S%. As a matter of fact, the results 
of Section 5 are also global in nature. Consider, for example, Proposition 
5.4. Fix a model and a state a and suppose that a = By (ri A---Arn). Since, 
for every formula y, B.y — B, Bx is a theorem of KD45ž, it follows that 
a H B,B,(r1A-+++Arn), that is, for every w € B,(a),w = Bu(ri A+++ Arn). 
Thus, it follows from Proposition 5.4 that o(w) € 9%, for every w € B,(a).'® 
That is, if at a state there is common belief of rationality, then at that state, 
as well as at all states reachable from it by the common belief relation By, 
it is true that the strategy profile played belongs to S°°. This is essentially 


16 This fact was proved directly in the proof of Proposition 5.4. 
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a global result, since from the point of view of a state a, the “global” space 
is precisely the set B,(a). 

Thus the only difference between the results of Section 5 and those of 
this section lies in the fact that Propositions 5.4 and 5.8 bring out the 
role of common belief by mimicking the informal argument that if player 
1 is rational then she won’t choose a strategy sı € DÌ and if player 2 
believes that player 1 is rational then he believes that s; ¢ D? and therefore 
will not choose a strategy s2 € D4, and if player 1 believes that player 2 
believes that player 1 is rational, then player 1 believes that s2 ¢ D} and 
will, therefore, not choose a strategy s; € D?, and so on. Beliefs about 
beliefs about beliefs...are explicitly modeled through the common belief 
operator. In contrast, Propositions 6.2 and 6.3 do not make use of the 
common belief operator. However, the logic is essentially the same. In 
particular, common belief of rationality is generated by the axiom WR’ 
(or SR’) and the rule of necessitation: from sẹ — —B,(s{ +1 s/) we get, 
by Necessitation, that Bi(s* —> =B,(s{ >1 s*)) A Bo(s? > =Bi(s{ >1 
s¥)) and thus, whatever is implied by WR’ is believed by both players. 
Further iterations of the Necessitation rule yields beliefs about beliefs about 
beliefs. .. about the rationality of every player. 


7 Related literature 


As noted in the introduction, the iterated elimination of strictly dominated 
strategies as a solution concept for strategic-form games goes back to Bern- 
heim [2] and Pearce [16] and has been further studied and characterized by 
a number of authors. From the point of view of this paper, the most im- 
portant contribution in this area is due to Stalnaker [17], who put forward 
the novel proposal of characterizing solution concepts for games in terms 
of classes of models. Stalnaker carried out his analysis within the standard 
framework of von Neumann-Morgenstern payoffs and defined dominance in 
terms of mixed strategies. Furthermore his analysis was semantic rather 
than syntactic. Our approach differs from Stalnaker’s in that we formulate 
rationality syntactically within an axiomatic system and provide character- 
ization results in line with the notion of frame characterization in modal 
logic. Furthermore, we do this in a purely ordinal framework that does not 
require probabilistic beliefs and von Neumann-Morgenstern payoffs. How- 
ever, our intellectual debt towards Stalnaker is clear. In particular, the 
IDIP algorithm (see Definition 3.4) is the adaptation to ordinal games of 
the algorithm he introduced in [17]. 

A syntactic epistemic analysis of iterated strict dominance was also pro- 
posed by de Bruin [9, p. 86]. However his approach is substantially dif- 
ferent from ours. First of all, his analysis is explicitly carried out only for 
two-person games, while we allowed for any number of players. Secondly, 
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de Bruin assumes von Neumann Morgenstern payoffs and his definition of 
strict dominance involves domination by mixed strategies [9, p. 51], while we 
considered ordinal payoffs and defined dominance in terms of pure strategies 
only (see Definition 3.2). Thirdly, de Bruin restricts attention to knowledge 
(that is, in his axiom system he imposes the Truth Axiom T; on individual 
beliefs: [9, p. 51]) and thus does not investigate the difference between the 
implications of common belief of rationality and those of common knowledge 
of rationality (hence in his analysis there is no counterpart to the difference 
highlighted in Propositions 5.4 and 5.8 of Section 5). More importantly, 
however, de Bruin introduces the notion of strict dominance directly into 
the syntax by using atomic propositions of the form nsd;(A;, Aj) whose in- 
tended interpretation is “player i uses a pure strategy in A; which is not 
strictly dominated by a mixed strategy over A; given that player j plays a 
pure strategy in A;”. Furthermore, his definition of rationality incorporates 
the notion of strict dominance. De Bruin’s definition of rationality [9, p. 86] 
consists of two parts: a basis step without knowledge, r; — nsd;(Aj;, A;), 
and an inductive step with knowledge: (r; ^ KiX; A KiX;) > nsdi(Xi, X;). 
According to de Bruin the advantage of his two-part definition of rationality 
is that 


Drawing a line between a basis case without beliefs, and an induc- 
tive step with beliefs makes it possible to mimic every single round 
of elimination of the solution concept by a step in the hierarchy of 
common belief in rationality. This becomes highly explicit in the 
inductive character of the proof. [9, p. 100] 


However, as pointed out at the end of the previous section, this mim- 
icking of the elimination steps occurs also in the proof of Proposition 5.4 
without the need for a two-part definition of rationality and, more impor- 
tantly, without incorporating the notion of dominance in the syntax. 

The disadvantage of de Bruin’s approach is that one loses the distinction 
between syntax and semantics and the ability to link the two by means of 
frame characterization results. In our analysis, the notion of strict domi- 
nance is purely a semantic notion, which has no syntactic counterpart. On 
the other hand the definition of rationality is expressed syntactically and it 
is epistemically based, in that it evaluates a player’s rationality by compar- 
ing her action with her beliefs about the desirability of alternative actions. 
The characterization results then establish a correspondence between the 
output of an algorithm (such as the iterated deletion of strictly dominated 
strategies) and common belief of an independently formulated notion of 
rationality. 

Borgers [7] provides a characterization of pure-strategy dominance that 
differs from ours. Like us, Borgers assumes that only the ordinal rankings of 
the players are commonly known; however—unlike us—he also assumes that 
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FIGURE 8. Two games in which player 2 has strategies d and e, whereas 
player 1 has either three strategies (left) or two (right). 


each player has a von Neumann-Morgenstern utility function on the set of 
outcomes, forms probabilistic beliefs about the opponents’ strategy choices 
and chooses a pure strategy that maximizes her expected utility, given those 
beliefs. He thus asks the question: what pure-strategy profiles are consistent 
with common belief of rationality, where the latter is defined as expected 
utility maximization with respect to some von Neumann-Morgenstern util- 
ity function and some beliefs about the opponents’ strategies? Borgers 
shows that a pure strategy is rational in this sense if and only if it is not 
dominated by another pure strategy. Thus there is no need to consider dom- 
inance by a mixed strategy. However, he shows that the relevant notion of 
dominance in this case is not strict dominance but the following stronger 
notion: a strategy s; € S; of player i is dominated if and only if, for every 
subset of strategy profiles X_; C S_; of the players other than i, there ex- 
ists a strategy t; € S; (which can vary with X_;) that weakly dominates s; 
relative to X_;.!" For example, in the game illustrated in Figure 8 (left), 
strategy a of player 1 is dominated (by b relative to {d,e} and also relative 
to {d} and by c relative to {e}). Thus in the corresponding model shown 
in Figure 9, at state a, while player 1 is rational according to our axiom 
WR (since no strategy is strictly dominated; indeed at state a there is 
common knowledge of rationality), she is not rational according to Bérgers’ 
definition.1® 

On the other hand, while stronger than the notion expressed by our 
axiom WR, Borgers’ notion of rationality is weaker than our axiom SR, as 
can be seen in the game of Figure 8 (right). Here strategy a of player 1 is 
dominated by b relative to {d,e} and also relative to {d} but not relative to 
{e}. Thus a is a rational strategy according to Borgers’ definition. On the 
other hand, in the model of Figure 9 (viewed now as a model for the game 
of Figure 8 (right)) player 1 is not rational at state a (where her choice is 


17 We say that t; weakly dominates s; relative to X_, if (1) z(t;,a_;) =i z(si,v_;), for 
all x_; E X-i, and (2) there exists an ĉ—; E€ X_,; such that z(t;,%_;) >i 2(si,€-:). 

18 The reason for this is as follows: if player 1 assigns positive probability to both a and 
B, then she would get a higher expected utility by switching to strategy b. The same 
is true if she assigns probability 1 to a. On the other hand, if she assigns probability 
1 to @ then she can increase her utility by switching to c. 
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FIGURE 9. A model for the games of Figure 8. 


a) according to the notion of rationality expressed by axiom SR (indeed, in 
this game, T® = { (a, e), (b, d)} and thus (a, d) ¢ T®). 


8 Conclusion 


We have examined the implications of common belief and common knowl- 
edge of two, rather weak, notions of rationality. Most of the literature 
on the epistemic foundations of game theory have dealt with the Bayesian 
approach, which identifies rationality with expected payoff maximization, 
given probabilistic beliefs (for surveys of this literature see [1] and [11]). 
Our focus has been on strategic-form games with ordinal payoffs and non- 
probabilistic beliefs. While most of the literature has been developed within 
the semantic approach, we have used a syntactic framework and expressed 
rationality in terms of syntactic axioms. We showed that the first, weaker, 
axiom of rationality characterizes the iterated deletion of strictly dominated 
strategies, while the stronger axiom characterizes the pure-strategy version 
of the algorithm introduced by Stalnaker [17]. 

The two notions of rationality used in this paper can, of course, be 
used also in the subclass of games with von Neumann-Morgenstern payoffs 
and the results would be the same. Furthermore, the standard notion of 
Bayesian rationality as expected payoff maximization is stronger than (that 
is, implies) both notions of rationality considered in this paper. Thus our 
results apply also to Bayesian rationality. 1° 

We have provided two versions of our characterization results. The 
first (Propositions 5.4 and 5.8), which comes closer to the previous game- 
theoretic literature, is based on an explicit account of the role of common 


19 In the sense that whatever is incompatible with our notion of rationality is also in- 
compatible with the stronger notion of Bayesian rationality. 
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belief of rationality and thus requires a syntax that contains atomic propo- 
sitions that are interpreted as “player i is rational”. The second charac- 
terization (Propositions 6.2 and 6.3) is closer to the modal logic literature, 
where axioms are characterized in terms of properties of frames. However, 
we argued that the two characterizations are essentially identical. 

We have restricted attention to strategic-form games. In future work 
we intend to extend this qualitative (that is, non probabilistic) analysis to 
extensive-form games with perfect information and the notion of backward 
induction. 
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Abstract 


We present an epistemic logic incorporating dynamic operators to 
describe information changing events. Such events include epistemic 
changes, where agents become more informed about the non-changing 
state of the world, and ontic changes, wherein the world changes. The 
events are executed in information states that are modelled as pointed 
Kripke models. Our contribution consists of three semantic results. 
(i) Every consistent formula can be made true in every information 
state by the execution of an event. (ii) Every event corresponds to 
an event with assignments to true and false only. (iii) Every event 
corresponds to a sequence of events with assignments of a single atom 
only. We apply the logic to model dynamics in a multi-agent setting 
involving card players. 


1 Introduction 


In dynamic epistemic logics [32, 23, 9, 4, 17] one does not merely describe 
the static (knowledge and) beliefs of agents but also dynamic features: how 
does belief change as a result of events taking place. The main focus of such 
logics has been change of only belief, whereas the facts describing the world 
remain the same. Change of belief is known as epistemic change. One can 
also model change of facts, and the resulting consequences of such factual 
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changes for the beliefs of the agents. Change of facts is also known as ontic 
change (change of the real world, so to speak).' In this contribution we use 
‘event’ to denote any sort of information change, both epistemic and ontic. 
Let us begin by a simple example involving various events. 


Example 1.1. Given are two players Anne and Bill. Anne shakes a cup 
containing a single coin and deposits the cup upside down on the table 
(there are no opportunities for cheating). Heads or tails? Initially, we have 
a situation wherein both Anne (a) and Bill (b) are uncertain about the 
truth of that proposition. A player may observe whether the coin is heads 
or tails, and/or flip the coin, and with or without the other player noticing 
that. Four example events are as follows. 


1. Anne lifts the cup and looks at the coin. Bill observes this but is not 
able to see the coin. All the previous is common knowledge to Anne 
and Bill. 


2. Anne lifts the cup and looks at the coin without Bill noticing that. 


3. Anne lifts the cup, looks at the coin, and ensures that it is tails (by 
some sleight of hand). Bill observes Anne looking but is not able to 
see the coin, and he considers it possible that Anne has flipped the 
coin to tails (and this is common knowledge). 


4. Bill flips the coin (without seeing it). Anne considers that possible 
(and this is common knowledge). 


Events 1, 3, and 4 are all public in the sense that the actual event is con- 
sidered possible by both agents, and that both agents know that, and know 
that they know that, etc.; whereas event 2 is private: Bill is unaware of the 
event; the event is private to Anne. Events 3 and 4 involve ontic change, 
whereas events 1 and 2 only involve epistemic change. Flipping a coin is 
ontic change: the value of the atomic proposition ‘the coin is heads’ changes 
from false to true, or from true to false, because of that. But in the case of 
events 1 and 2 that value, whether true or false, remains unchanged. What 
changes instead, is how informed the agents are about that value, or about 
how informed the other agent is. In 1 and 2, Anne still learns whether the 
coin is heads or tails. In 1, Bill ‘only’ learns that Anne has learnt whether 
the coin is heads or tails: he has not gained factual information at all. In 
Example 2.5, later, we will formalize these descriptions. 


1 In the areas known as ‘artificial intelligence’ and ‘belief revision’, epistemic and ontic 
change are called, respectively, belief revision [1] and belief update [27]. We will not 
use that terminology. 
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Various logics have been proposed to model such events. A well-known 
setting is that of interpreted systems by Fagin et al. [21]. Each agent has 
a local state; the local states of all agents together with a state of the 
environment form a global state; belief of an agent is modelled as uncertainty 
to distinguish between global states wherein the agent has the same local 
state, and change of belief is modelled as a transition from one global state 
to another one, i.e., as a next step in a run through the system. In an 
interpreted system the treatment of epistemic and ontic change is similar— 
either way it is just a next step in a run, and how the valuation between 
different points changes is not essential to define or describe the transition. 
There is a long tradition in such research [30, 21]. 

The shorter history of dynamic epistemic logic started by focusing on 
epistemic change [32, 23, 9, 4, 17]. In that community, how to model ontic 
change was first mentioned by Baltag, Moss, and Solecki as a possible ex- 
tension to their action model logic for epistemic change [5]. More detailed 
proposals for ontic change are far more recent [20, 16, 15, 10, 28, 33, 25, 26]. 
The literature will be discussed in more detail in Section 5. 


Section 2 contains logical preliminaries, including detailed examples. 
Section 3 contains the semantic results that we have achieved for the logic. 
This is our original contribution to the area. These results are that: (i) for 
all finite models and for all consistent formulas we can construct an event 
that ‘realizes’ the formula, i.e., the formula becomes true after execution of 
the event; that: (ii) every event (with assignments of form p := y, for y in 
the language) corresponds to an event with assignments to true and false 
only; and also that: (iii) every event corresponds to a sequence of events 
with assignments for a single atom only. Section 4 applies the logic to model 
the dynamics of card players. Section 5 discusses related work in detail. 


2 A logic of ontic and epistemic change 


We separately introduce the logical language, the relevant structures, and 
the semantics of the language on those structures. The syntax and semantics 
appear to overlap: updates are structures that come with preconditions that 
are formulas. In fact it is properly covered by the double induction used 
in the language definition, as explained after Definition 2.4 below. For a 
detailed treatment of the logic without ontic change, and many examples, 
we recommend [17]; for more examples of the logic involving ontic change, 
see [15, 16, 20]. 


2.1 Language 


We use the style of notation from propositional dynamic logic (PDL) for 
modal operators which is also used in [10]. 
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Definition 2.1 (Language). Let a finite set of agents A and a countable 
set of propositional variables P be given. The language X is given by the 
following BNF: 

p = p| wl erg | tale 

a ::= a | B* | (U,e) 
where p € P, a € A, B C A (the dynamic operator [B*] is associated with 
‘common knowledge among the agents in B’), and where (U, e) is an update 
as (simultaneously) defined below. 


We use the usual abbreviations, and conventions for deleting parenthe- 
ses. In particular, [B]y stands for Aac ,|a]y, and (the diamond form) (a)y 
is equivalent to =[a]>y. Non-deterministic updates are introduced by ab- 
breviation: [(U,e) U (U’, f)]y is by definition [U, ely A [U’, fly. 


2.2 Structures 


Epistemic model. The models which adequately present an information 
state in a multi-agent environment are Kripke models from epistemic logic. 
The set of states together with the accessibility relations represent the in- 
formation the agents have. If one state s has access to another state t for 
an agent a, this means that, if the actual situation is s, then according to 
a’s information it is possible that t is the actual situation. 


Definition 2.2 (Epistemic model). Let a finite set of agents A and a count- 
able set of propositional variables P be given. An epistemic model is a triple 
M = (S, R, V) such that 


e domain S is a non-empty set of possible states, 
e R: A (Sx S) assigns an accessibility relation to each agent a, 


e V : P > p(S) assigns a set of states to each propositional variable; 
this is the valuation of that variable. 


A pair (S, R) is called an epistemic frame. A pair (M,s), with s € S, is 
called an epistemic state. 


A well-known notion of sameness of epistemic models is ‘bisimulation’. 
Several of our results produce models that are bisimilar: they correspond 
in the sense that even when not identical (isomorphic), they still cannot be 
distinguished in the language. 


Definition 2.3 (Bisimulation). Let two models M = (5,R,V) and M’ = 
(S’, R’, V") be given. A non-empty relation R C S$ x S$” is a bisimulation iff 
for all s € S and s’ € S’ with (s,s’) € R: 


atoms for all p € P: s € V(p) iff s’ € V'(p); 
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forth for all a € A and all t € S: if (s,t) € R(a), then there is a t € S” 
such that (s’,t’) € R’(a) and (t,t’) € R; 


back for alla € A and all t’ € S’: if (s’,t’) € R'(a), then there isate S 
such that (s,t) € R(a) and (t,t) E€ R. 


We write (M,s) = (M’,s’), iff there is a bisimulation between M and 

M” linking s and s’, and we then call (M, s) and (M’, s’) bisimilar. A model 
such that all bisimilar states are identical is called a bisimulation contraction 
(also known as a strongly extensional model). 
Update model. An epistemic model represents the information of the 
agents. Information change is modelled as changes of such a model. There 
are three variables. One can change the set of states, the accessibility rela- 
tions and the valuation. It may be difficult to find the exact change of these 
variables that matches a certain description of an information changing 
event. It is often easier to think of such an event separately. One can model 
an information changing event in the same way as an information state, 
namely as some kind of Kripke model: there are various possible events, 
which the agents may not be able to distinguish. This is the domain of 
the model. Rather than a valuation, a precondition captures the conditions 
under which such events may occur, and postconditions also determine what 
epistemic models may evolve into. Such a Kripke model for events is called 
an update model, which were first studied by Baltag, Moss and Solecki, and 
extended with simultaneous substitutions by van Eijck [5, 20].? Here we use 
van Eijck’s definition. 


Definition 2.4 (Update model). An update model for a finite set of agents 
A and a language Y is a quadruple U = (E, R, pre, post) where 


e domain E is a finite non-empty set of events, 
e R:A— e(E x E) assigns an accessibility relation to each agent, 
e pre: E— Z assigns to each event a precondition, 


e post: E > (P — Z) assigns to each event a postcondition for each 
atom. Each post(e) is required to be only finitely different from the 
identity id; the finite difference is called its domain dom(post(e)). 


A pair (U,e) with a distinguished actual event e € E is called an update. 
A pair (U,E) with E’ C E and |E’| > 1 is a multi-pointed update, first in- 
troduced in [19]. The event e with pre(e) = T and post(e) = id we name 


2 In the literature update models are also called action models. Here we follow [10] and 
call them update models, since no agency seems to be involved. 
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skip. An update with a singleton domain, accessible to all agents, and pre- 
condition T, is a public assignment. An update with a singleton domain, 
accessible to all agents, and identity postcondition, is a public announce- 
ment. 


Instead of 
pre(e) = y and post(e)(p1) = Y1,- .., and post(e)(Pn) = Yn 
we also write? 
for event e: if y, then pı := ¥,..., and Pn := Yn 


The event skip stands for: nothing happens except a tick of the clock. 

To see an update as part of the language we observe that: an update 
(U, e) is an inductive construct of type a that is built the frame underlying U 
(we can assume a set enumerating such frames) and from simpler constructs 
of type y, namely the preconditions and postconditions for the events of 
which the update consists. This means that there should be a finite number 
of preconditions and a finite number of postconditions only, otherwise the 
update would be an infinitary construct. A finite number of preconditions is 
guaranteed by restricting ourselves in the language to finite update models. 
A finite number of postconditions is guaranteed by (as well) restricting 
ourselves to finite domain for postconditions. This situation is similar to 
the case of automata-PDL [24, Chapter 10, Section 3]. 

If in case of nondeterministic updates the underlying models are the 
same, we can also see this as executing a multi-pointed update. For example, 
(U,e) U (U, f) can be equated with (U, {e, f}). 


Example 2.5. Consider again the scenario of Example 1.1 on page 88. Let 
atomic proposition p stand for ‘the coin lands heads’. The initial information 
state is represented by a two-state epistemic model with domain {1,0}, with 
universal access for a and b, and with V(p) = {1}. We further assume that 
the actual state is 1. This epistemic state is depicted in the top-left corner 
of Figure 1. The events in Example 1.1 can be visualized as the following 
updates. The actual state is underlined. 


1. Anne lifts the cup and looks at the coin. Bill observes this but is not 
able to see the coin. All the previous is common knowledge to Anne 


and Bill. 
7 oN ON 
a,b p —b— np a,b 
NAi NY 


3 The notation is reminiscent of that for a knowledge-based program in the interpreted 
systems tradition. We discuss the correspondence in Section 5. 
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Here, pre(p) = p, post(p) = id, pre(np) = —p, post(np) = id. The 
update model consists of two events. The event p corresponds to 
Anne seeing heads, and the event np to Anne seeing tails; Anne is 
aware of that: thus the reflexive arrows (identity relation). Bill cannot 
distinguish them from one another: thus the universal relation. The 
aspect of common knowledge, or common awareness, is also present 
in this dynamic structure: the reflexive arrow for Anne also encodes 
that Anne knows that she lifts the cup and that Bill observes that; 
similarly for Bill, and for iterations of either awareness. 


2. Anne lifts the cup and looks at the coin without Bill noticing that. 


A/m Ji 
a pb skip a,b 
Ne CA 


Event p is as in the previous item and skip is as above. In this update, 
there is no common awareness of what is going on: Anne observes 
heads knowing that Bill is unaware of that, whereas Bill does not 
consider the actual event; the b-arrow points to the other event only. 


3. Anne lifts the cup, looks at the coin, and ensures that it is tails (by 
some sleight of hand). Bill observes Anne looking but is not able to 
see the coin, and he considers it possible that Anne has flipped the 
coin to tails (and this is common knowledge). 


ZTN EEN 
a,b p —b— np a,b 
NA PA NY 

b b 
N Z 
a,b p’ 
Na 


Events p and np are as before, whereas pre(p’) = T, post(p’)(p) = L. 
The event p’ may take place both when the coin is heads and when 
the coin is tails, in the first case atom p is set to false (tails), and in 
the second it remains false. 


4. Bill flips the coin (without seeing it). Anne considers that possible 
(and this is common knowledge). 


) 


ETN 
a,b n <— a — skip a,b 


NW ~Y/ 
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Here, pre(n) = T, post(n)(p) = ~p, and skip is as before. For models 
where all accessibility relations are equivalence relations we will also 
use a simplified visualization that merely links states in the same 
equivalence class. E.g., this final event is also depicted as: 


n — a — skip 


2.3 Semantics 

The semantics of this language is standard for epistemic logic and based 
on the product construction for the execution of update models from the 
previous section. Below, R(B)* is the transitive and reflexive closure of the 
union of all accessibility relations R(a) for agents a € B. Definitions 2.6 
and 2.7 are supposed to be defined simultaneously. 


Definition 2.6 (Semantics). Let a model (M,s) with M = (S,R,V) be 
given. Let a € A, BCA, and y, we Z. 


(M,s) =p if seV(p) 

(M, s) = =% iff (M, s) Fy 

(M,s)EpAw iff (M,s) Ey and (M, s) Ew 

(M,s) H [aly iff (M,t) E ¢ for all t such that (s,t) € R(a) 
(M,s)—[B*]y~ if (M,t) — ¢ for allt such that (s,t) € R(B)* 
(M,s)—[U,ely iff (1,5) — pre(e) implies (M & U, (s,e)) Ey 


We now define the effect of an update on an epistemic state—Figure 1 
gives an example of such update execution. 


Definition 2.7 (Execution). Given are an epistemic model M = (S, R, V), 
a state s E€ S, an update model U = (E,R, pre, post), and an event e € E 
with (M, s) — pre(e). The result of executing (U,e) in (M, s) is the model 
(M @ U, (s,e)) = ((S’, R', V”), (s,e)) where 
e S’ = {(t,f) | (M,t) = pre(f)}, 
e R'(a) = {((t,f), (ug) | (tf), (ug) E€ S” and (t,u) € R(a) and 
(f,g) E€ R(a)}, 


© V'(p) = {(t,f) | (M, t) K post(f)(p)} 


Definition 2.8 (Composition of update models). Let update models U = 
(E, R, pre, post) and U’ = (E’,R’, pre’, post’) and events e € E and e’ € E’ be 
given. The composition (U,e) ; (U’,e’) of these update models is (U”, e”) 
where U” = (E”, R”, pre”, post”) is defined as 


e EV =EXE’, 
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aN 
a,b 1 << a,b — 0 a,b 4 ) 

N44 YY (1, p) 

7N 
X = b b 
/ N 
N a,b (l,skip) —— b — (0,skip) a,b 

a P b —— skip a,b SA \Y 

NX NY 


FIGURE 1. In an epistemic state where Anne and Bill are uncertain about 
the truth of p (heads or tails), and wherein p is true, Anne looks at the coin 
without Bill noticing it. The actual states and events are underlined. 


e R’(a) = {((f, f"), (g,8’)) | (fg) € R(a) and (f',g') € R'(a)}, 
e pre”(f, f’) = pre(f) A [U, f]pre’(f’), 


e dom(post” (f, f’)) = dom(post(f)) U dom(post’(f’)) and 
if p € dom(post” (f, f’)), then 


t(f ifpgd t (f 
post” (f, f^) (p) = post( )(p) i fpg otniPes (f")), 
[U, f]post’(f’)(p) otherwise. 
The reason for [U, f]post’(f’)(p) in the final clause will become clear from 
the proof detail shown for Proposition 2.9. 


Proposition 2.9. } [U,e][U’,e’]y = [(U,e) ; (U’,e’)]y 


Proof. Let (M,t) be arbitrary. To show that (M,t) H} [(U,e) ; (U’,e’)]y if 
and only if (M,t) H [U,e][U’, e’]y, it suffices to show that M @(U; U’) is 
isomorphic to (M @U)@U’. A detailed proof (for purely epistemic updates) 
is found in [17]. The postconditions (only) play a part in the proof that the 
valuations correspond: 

For the valuation of facts p in the domain of post” we distinguish the 
cases (p E€ dom(post(e)) but p ¢ dom(post’(e’))) (i), and otherwise (ii). 
The valuation of a i-atom in a triple (t,(e,e’)) is post(e)(p) according to 
the definition of updates composition; and the valuation of a ii-atom is 
[U, e]post’(e’)(p). Consider the corresponding triple ((t,e),e’). The valu- 
ation of an i-atom in (t,e) is post(e)(p), and because p does not occur 
in dom(post’(e’)) its value in the triple ((t,e),e’) will remain the same. 
For a ti-atom, its final value will be determined by evaluating post’ (e’) (p) 
in ((M & U), (t,e)). This corresponds to evaluating [U, e]post’(e’)(p) in 
(M,t). Q.E.D. 
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2.4 Proof system 


A proof system UM for the logic is given in Table 1. The proof system 
is a lot like the proof system for the logic of epistemic actions (i.e., for 
the logic without postconditions to model valuation change) in [5]. There 
are two differences that makes it worthwhile to present this system. The 
axiom ‘atomic permanence’ in [5]—[U, e]p < (pre(e) — p)—is now instead 
an axiom expressing when atoms are not permanent, namely how the value 
of an atom can change, according to the postcondition for that atom: 


[U, e]p > (pre(e) — post(e)(p)) update and atoms 
The second difference is not apparent from Table 1. The axiom 
[U, e][U’, e] > [(U,e) ; (U’,e’)]yp update composition 


also occurs in [5]. But Definition 2.8 to compute that composition is in our 
case a more complex construction than the composition of update models 
with only preconditions, because it also involves resetting the postcondi- 
tions. We find it remarkable that these are the only differences: the inter- 
action between postconditions for an atom and the logical operators, only 
occurs in the axiom where that atom is mentioned, or implicitly, whereas 
the interaction between preconditions and the logical operators appears in 
several axioms and rules. 

The proof system is sound and complete. The soundness of the ‘update 
and atoms’ axiom is evident. The soundness of the ‘update composition’ 
axiom was established in Proposition 2.9. We proved completeness of the 
logic as a modification of the completeness proof for the logic without ontic 
change—action model logic—as found in [17], which in turn is a simplified 
version of the original proof for that logic as found in [5]. We do not consider 
the modified proof of sufficient original interest to report on in detail. 


3 Semantic results 


We now present some semantic peculiarities of the logic. These we deem 
our contribution to the area. The results help to relate different approaches 
combining ontic and epistemic change (see Section 5). The various ‘normal 
forms’ for update models that we define are also intended to facilitate future 
tool development. Finally, they are relevant when modelling AGM belief 
revision [1] in a dynamic epistemic setting. 


3.1 Arbitrary belief change 


Let (M, s) and (M’, s") be arbitrary finite epistemic states for the same set of 
atoms and agents, with M = (S,R,V) and M’ = (S’, R’,V’). Surprisingly 
enough, there is almost always an update transforming the former into the 
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All instantiations of propositional tautologies 

al(p > 4) > (laly > laly) distribution 

From y and y —> 4, infer 4% modus ponens 

From y, infer [a]y necessitation 

U,eļp = (pre(e) — post(e)(p)) update and atoms 
update and negation 
update and conjunction 
update and knowledge 
update composition 
mix 

induction axiom 


Let (U,e) be an update model and let a set updates and common 
of formulas xç for every f such that (e,f) € knowledge 

R(B)* be given. From ys — [U,f]y and 

(xe A pre(f)) — [a]Xg for every f € E such 

that (e,f) € R(B)*, a € B and (f,g) € 

R(a), infer Xe > [U, e][B*]y. 


TABLE 1. The proof system UM. 


latter. There are two restrictions. Both restrictions are technical and not 
conceptual. Firstly, for the set of agents with non-empty access in M” there 
must be a submodel of M containing actual state s that is serial for those 
agents. In other words, if an agent initially has empty access and therefore 
believes everything (‘is crazy’) you cannot change his beliefs, but otherwise 
you can. This seems reasonable. Secondly, models M and M’ should only 
differ in the value of a finite number of atoms; more precisely, if we define 
that 


an atom is relevant in a model iff its valuation is neither empty nor 
the entire domain, 


then the requirement is that only a finite number of atoms is relevant in 
MUM". This is required, because we can only change the value of a finite 
number of atoms in the postconditions. This also seems reasonable: as both 
models are finite, the agents can only be uncertain about the value of a finite 
number of atoms (in the combined models M and M’); in other words, they 
are ‘not interested’ in the value of the remaining atoms. 

For expository purposes we initially assume that all agents consider the 
actual state s of M a possibility (as in all S5 models, such as Kripke models 
representing interpreted systems), thus satisfying the first of the two re- 
strictions above: the serial submodel required is then the singleton model 
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consisting of s, accessible to all agents. The update transforming (M, s) into 
(M’, s’) can be seen as the composition of two intuitively more appealing 
updates. That will make clear how we can also describe the required update 
in one stroke. 

In the first step we get rid of the structure of M. As the epistemic state 
(M, s) is finite, it has a characteristic formula 6(,s) [6, 8].4 We let the 
agents publicly learn that characteristic formula. This event is represented 
by the singleton update (U,e) defined as 


(({e}, R, pre, post), e) with pre(e )= (M,s)> 
for all a: (e,e) € R(a), 
and post(e) = id 


In other words, the structure of the current epistemic state is being publicly 
announced. The resulting epistemic state is, of course, also singleton, or 
bisimilar to a singleton epistemic state, as 4(,7,5) holds in all states in M 
bisimilar to s. Without loss of generality assume that it is singleton. Its 
domain is {(s,e)}. This pair (s,e) is accessible to itself because for all 
agents, (s,s) € R(a) (all agents consider the actual state s a possibility), and 
(e,e) € R(a). The valuation of propositional variables in this intermediate 
state are those of state s in M. What the value is does not matter: we will 
not use that valuation. 

Now proceed to the second step. In the epistemic state wherein the 
agents have common knowledge of the facts in s, the agents learn the struc- 
ture of the resulting epistemic state M’ = (S’, R’,V’) and their part in it 
by executing update (U’, s’) defined as 


((S', R’, pre’, post’), s’) with for all t € S’: pre’(t’) = T and 
for relevant p : post’(t’)(p) = T iff t € V’(p) 


Note that the domain S’ and the accessibility relation R’ of U’ are precisely 
those of M’, the resulting final epistemic model. The postcondition post’ 
is well-defined, as only a finite number of atoms (the relevant atoms) is 
considered. Because we execute this update in a singleton model with public 
access, and because it is executable for every event t’, the resulting epistemic 
state has the same structure as the update: it returns S’ and R’ again. The 
postcondition delivers the required valuation of atoms in the final model: 
for each event t’ in U’ and for all relevant atoms p, p become true in t if p 
is true in state t’ in M’ (post(t’)(p) = T), else p becomes false. The value 
of irrelevant atoms remains the same. 
4 A characteristic formula y for a state (M, s) satisfies that for all y, (M,s) = ọ iff 
y H| w. In fact, for the construction we only need formulas that can distinguish all 


states in the domain from one another, modulo bisimilarity. Characteristic formulas 
satisfy that requirement. 
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We combine these two updates into one by requiring the precondition of 
the first and the postcondition of the second. Consider U/. that is exactly 
as U’ except that in all events t’ in its domain the precondition is not T but 
(ms): the characteristic formula of the point s of (M,s). Update (U;., s”) 
does the job: epistemic state (M @U’, (s, s’)) is isomorphic to (M’, s’). This 
will be Corollary 3.3 of our more general result, to follow. 


Now consider the more general case that the agents with non-empty 
access in M’ are serial in a submodel M** of M that contains s, with 
domain S**. In other words: at least all agents who finally have consistent 
beliefs in some states, initially have consistent beliefs in all states. The 
construction above will no longer work: if the actual state is not considered 
possible by an agent, then that agent has empty access in actual state (s, s”) 
of (M @ U}), but not in s’ in M’. But if we relax the precondition 6(1,5), 
for the point s of (M,s), to the disjunction Vess Ô(M,u), that will now 
carry along the serial subdomain SS, the construction will work because 
an agent can then always imagine some state wherein the update has been 
executed, even it that is not the actual state. This indeed completes the 
construction. 


Definition 3.1 (Update for arbitrary change). Given finite epistemic mod- 
els M = (S,R,V) and M’ = (9', R', V’) for the same sets of agents and 
atoms. Assume that all agents with non-empty access in M’ are serial in 
M5“ containing s. The update for arbitrary change (U”, (s, s’)) = ((E”, R”, 
pre”, post”), (s, s’)) is defined as (for arbitrary agents a and arbitrary rele- 
vant atoms p): 


E” = Ss’ 
(tu) E€ R”(a) if (t'u) € R'(a) 
pre” (t) = V ÔM, u) 
we Sse 
T dee EV? 
post”(t’)(p) = ee 
L otherwise 


The epistemic state (M & U”,(s,s’)) is bisimilar to (M’, s’), which is 
the desired result. It will typically not be isomorphic: M & U” can be seen 
as consisting of a number of copies of M’ (namely |S**"| copies) ‘with the 
accessibility relations just right to establish the bisimulation’. One copy 
may not be enough, namely when the state t in M to which that copy 
corresponds, lacks access for some agents. This access will then also be 
‘missing’ between the states of ({t} x S’). But because of seriality one of 
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the other M’ copies will now make up for this lack: there is a u € S** such 
that (t,u) € R(a), which will establish access when required, as in the proof 
of the following proposition. 


Proposition 3.2. Given (M, s), (M’,s’), and U” as in Definition 3.1. Then 
R: ((M 8 U”), (s,s')) e (M',s') by way of, for all t € S* and t € S’: 
R(t, t’) =Y. 


Proof. Let R® be the accessibility relation and V® the valuation in (M @ 
U”). 

atoms: For an arbitrary relevant atom p: (t,t’) € V®(p) iff (M,t) H 
post” (t')(p), and by definition of post” we have that (M, t) = post” (t')(p) 
iff t € V'(p). Irrelevant atoms do not change value. 

forth: Let ((t1,t4),(t2,th)) € R®(a) and ((t1, t1), ti) € R. From 
((t1,t4), (t2,t4)) € R8 (a) follows (t{,t4) € R'(a). By definition of R we 
also have ((t2,t5), t2) E R. 

back: Let ((t1,t4),t,) E€ R and (t4,t4) € R'(a). As M5¥ is serial for 
and tı E€ S57, there must be a tz such that (t1,t2) € R(a). As (M, t2) 
Vicdom(m=r) ôcm,t) (because t2 is one of those t) we have that (t2, t4) 
dom(M & U”). From that, (t1,t2) € R(a), and (t1, t2) € R’(a), follows that 
((t1, t1), (t2, t4)) E€ RS (a). By definition of R we also have ((t2, th), th) E€ R. 

Q.E.D. 


Q 


MT 


Note that we keep the states outside the serial submodel M** out of the 
bisimulation. Without the seriality constraint the ‘back’ condition of the 
bisimilarity cannot be shown: given a ((t1, t1), t1) E€ R and (t4,t4) € R’(a), 
but where tı has no outgoing arrow for a, the required a-accessible pair 
from (t1, t1) does not exist. A special case of Proposition 3.2 is the corollary 
already referred to during the initial two-step construction, that achieves 
even isomorphy: 


Corollary 3.3. Given (M, s), (M’,s’), and U/, as above. Assume that M 
is a bisimulation contraction. Then (M @ UL) = M’. 


Proof. In this special case we have that (t,t’) € dom(M & U!) iff (M, t) = 
pre’(t’) iff (M,t) H| 6(,s) for the point s of (M, s). As the last is only the 
case when t = s (as M is a bisimulation contraction), we end up with a 
domain consisting of all pairs (s,t’) for all t € S’, a 1-1-correspondence. 
The bisimulation A above becomes the isomorphism 3(s,t’) = t. Q.E.D. 


A different wording of Proposition 3.2 is that for arbitrary finite epis- 
temic states (M,s) and (M’,s’) also satisfying the serial submodel con- 
straint, there is an update (U,e) transforming the first into the second. A 
final appealing way to formulate this result is: 
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Corollary 3.4. Given are a finite epistemic state (M, s) and a satisfiable 
formula y. If all agents occurring in p have non-trivial beliefs in state s 
of M, then there is an update realizing y, i.e., there is a (U,e) such that 
(M,s) = (U,e)y. 


Using completeness of the logic, this further implies that all consistent 
formulas can be realized in any given finite model. We find this result both 
weak and strong: it is strong because any conceivable (i.e., using the same 
propositional letters and set of agents) formal specification can be made 
true whatever the initial information state. At the same time, it is weak: 
the current information state does apparently not give any constraints on 
future developments of the system, or, in the opposite direction, any clue 
on the sequence of events resulting in it; the ability to change the value of 
atomic propositions arbitrarily gives too much freedom. Of course, if one 
restricts the events to specific protocols (such as legal game moves [15], and 
for a more general treatment see [11]), the amount of change is constrained. 
AGM belief revision and belief update. Our results on arbitrary belief 
change seem related to the postulate of success in AGM belief revision [1]. 
AGM belief revision corresponds to epistemic change, and AGM (in their 
terminology) belief update [27] corresponds to ontic change (unfortunately, 
in the AGM community ‘update’ means something far more specific than 
what we mean by that term). Given this correspondence we can achieve only 
expansion by epistemic change, and not proper revision; and the combina- 
tion of ontic and epistemic change can be seen as a way to make belief update 
result in belief revision. Apart from this obvious interpretation of epistemic 
and ontic change, one can also view our result that all consistent formulas 
can be realized, differently: a consequence of this is that for arbitrary con- 
sistent y and w there is an update (U,e) such that [aly — (U,e)[a]y. In 
AGM terms: if ọ is believed, then there is a way to revise that into belief of 
w, regardless of whether y A w is consistent or not. In other words: revision 
with w is always successful. That suggests that our way of achieving that 
result by combining epistemic and ontic change might somehow simulate 
standard AGM belief revision. Unfortunately it is immediately clear that 
we allow far too much freedom for the other AGM postulates to be fulfilled. 
It is clearly not a minimal change, for example. So walking further down 
this road seems infeasible. 


3.2 Postconditions true and false only 


The postconditions for propositional atoms can be entirely simulated by 
the postconditions true or false for propositional atoms. For a simple exam- 
ple, the public assignment p := y can be simulated by a two-point update 
e——A——f (ie., a nondeterministic event where all agents in A cannot 
distinguish e from f) such that pre(e) = y, post(e)(p) = T, pre(f) = -y, 
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post(f)(p) = L. In the public assignment (p := p,q := Y) to two atoms p 
and q we would need a four-point update to simulate it, to distinguish all 
four ways to combine the values of two independent atoms. 

The general construction consists of doing likewise in every event e of 
an update model. For each e we make as many copies as the cardinality of 
the powerset of the range of the postcondition associated with that event. 
Below, the set {0,1}4°™est()) represents that powerset. 


Definition 3.5 (Update model U'). Given is an update model U = (E, 
R, pre, post). Then U = (E™,R™, pre", post '+) is a normal update model 
with 


© E™ = Uncet(e, f) | f € {0, Idomros))} 
e ((e,f),(e, f’)) € R™ (a) iff (e,e’) € R(a) 


e pre™(e, f) = prele) A A ftp)=1 Post(e)(p) A Ajip)~0 =Post(e) (p) 


i if f(p) =1 


e post (e, f)(p) = 4 | if f(p) =0 


Proposition 3.6. Given an epistemic model M = (S, R, V) and an update 
model U = (E, R, pre, post) with normal update model U™ defined as above. 
Then (M 8 U) &(M@U"). 


Proof. We show that the relation % : (M & U) = (M 8 UH) defined as 


((s,e),(s,e, f)) € R iff (M, s) H pre ™ (e, f) 


is a bisimulation. Below, the accessibility relations in (M&U) and (M@U") 
are also written as R(a). 


atoms 
Let (s,e, f) be a state in the domain of (M @U"). We have to show 
that for all atoms p, (M, s) = post(e)(p) — post '“(e, f)(p). From the 
definition of post it follows that 


post ™(e, f)(p) iff f(p) =1. 
From (M, s) | pre™ (e, f) and the definition of pre follows that 


(M, s) = post(e)(p) iff f(p)=1. 


Therefore 
(M, s) = post(e)(p) > post '*(e, f)(p) . 
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forth 
Assume that ((s,e), (s’,e’)) € R(a) and that ((s,e),(s,e, f)) E R. Let 
f’ : dom(post(e’)) > {0, 1} be the function eh that 


ae f if (M, s’) H post(e’)(p) 


1 
fp 0 otherwise 


Therefore (M, s’) | pret (e', f’). Therefore ((s’,e’), (s’,e’, f’)) E€ R. 
From ((s,e),(s’,e’)) € Refa ) follows (s,s’) € R(a) and (e,e’) € 
R(a). From (e,e’) € R(a) and the definition of access R follows 
(e P), (€, F) € RE (a). From (s,s) € R(a) and ((e, f), (e, F) € 
R" (a) follows ((s,e, f), (s’,e’, f’)) € R(a). 


back 


Suppose ((s,e), (s,e, f)) € R and (s,e, f), (s’,e’, f’) € R(a). From the 
last follows (s, s’) € R(a) and ((e, f), (e’, f’)) € RH (a), therefore also 
(e,e’) € R(a). Therefore ((s, e), (s,e’)) € R(a). as in the case of 
forth it is established that ((s', e’), (s',e, fE 


Q.E.D. 


Corollary 3.7. The logic of change with postconditions true and false only 
is equally expressive as the logic of change with arbitrary postconditions. 


Although it is therefore possible to use postconditions true and false only, 
this is highly unpractical in modelling actual situations: the descriptions of 
updates become cumbersome and lengthy, and lack intuitive appeal. 

A transformation result similar to that in Proposition 3.6 can not be 
established for the logic with only singleton update models, i.e., the logic of 
public announcements and public assignments (as in [28]). If public assign- 
ments could only be to true and to false, then updates with assignments 
always result in models wherein the assigned atoms are true throughout the 
model, or false throughout the model. Therefore, there is no transforma- 
tion of, e.g., p——p into p———7p using public assignments and public 
announcements only. The construction above results in a two-event update 
model, that is not a singleton. 

A transformation result as in Proposition 3.6 immediately gives an ex- 
pressivity result as in Corollary 3.7 for the languages concerned. It is also 
tempting to see such a transformation result as a different kind of expres- 
sivity result. In two-sorted languages such as the one we consider in this 
paper one can then distinguish between the expressivity of two kinds of syn- 
tactic objects. A formula (p) corresponds to a class of models that satisfy 
that formula, and a modality (œ) corresponds to a relation on the class of 
models. The result is stated in terms of the expressivity of formulas, but 
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it is also a result about the expressivity of modalities. These two kinds of 
expressivity are not necessarily linked. One can have logics with the same 
expressivity of formulas, that have different expressivity of modalities and 
vice versa. 


3.3 Single assignments only 


Consider the update model e a f for a single agent a and for two 
atoms pı and p2 such that in e, if yı then pı := y2 and pə := p3, and in f, 
if p4 then pı := ys and p2 := ye. Can we also do the assignments one by 
one? In other words, does this update correspond to a sequence of updates 
consisting of events g in which at most one atom is assigned a value: the 
cardinality of dom(g) is at most 1. This is possible! First we ‘store’ the value 
(in a given model (M, s) wherein this update is executed) of all preconditions 
and postconditions in fresh atomic variables, by public assignments. This 
can be in arbitrary order, so we do it in the order of the y;. This is the 
sequence of six public assignments q1 := Y1, q2 := P2, G3 := $3, q4 := Ya, 
ds := $5, and gg := Ye. Note that such public assignments do not change the 
structure of the underlying model. Next we execute the original update but 
without postconditions. This is e’ a f’ with pre(e’) = pre(e) = 1 
and pre(f’) = pre(f) = p4 and with post(e’) = post(f’) = id. Note that 
qı remains true whenever e’ was executed, because qı was set to be true 
whenever yı was true, the precondition of both e and e. Similarly, q4 
remains true whenever f’ was executed. We have now arrived at the final 
structure of the model, just not at the proper valuations of atoms. 

Finally, the postconditions are set to their required value, conditional to 
the execution of the event with which they are associated. Agent a must 
not be aware of those conditions (the agent cannot distinguish between e’ 
and f’). Therefore we cannot model this as a public action. The way out of 
our predicament is a number of two-event update models, namely one for 
each postcondition of each event in the original update. One of these two 
events has as its precondition the fresh atom associated with an event in 
the original update, and the other event its negation, and agent a cannot 
distinguish between both. The four required updates are 


e ey—a—e} with in e1, if qı then po := q2 and in ej, if ~qı then id 
e e—a—e, with in eg, if qı then p3 := q3 and in e5, if ~qı then id 
e e3—a—e, with in es, if q4 then ps := qs and in e}, if —~q4 then id 
e ey—a—e, with in eg, if q4 then pe := ge and in e}, if —~q4 then id 


Now, we are done. These four final updates do not change the structure of 
the model, when executed. Therefore, now having set the postconditions 
right, the composition of all these constructs is isomorphic to the original 


Ontic and Epistemic Change 105 


update model! The general construction is very much as in this simple 
example. 


Definition 3.8 (Update model U°"*). Given an update model U = (E,R, 
pre, post), update model U°"* is the composition of the following update 
models: First perform Uece|dom(post(e)) + 1| public assignments for fresh 
variables g,..., namely for each e € E, g§ := pre(e), and for all p1,..., pn € 
dom(post(e)), gf := post(e)(p1),..., q := post(e)(pn). Then execute U but 
with identity (‘trivial’) postconditions, i.e., execute U’ = (E,R, pre, post’) 
with post’(e) = id for alle € E. Finally, execute Nece|dom(post(e))| two- 
event update models with universal access for all agents wherein for each 
event just one of its postconditions is set to its required value, by way of the 
auxiliary atoms. For example, for e € E as above the first executed update 
is e; ——A——ey with in e1, if q8, then pı := qf, and in eg, if 4q5 then id. 


The following proposition will be clear without proof: 


Proposition 3.9. Given epistemic model M and update model U exe- 
cutable in M. Then U°"* is isomorphic to U, and (M @ U°"®) is isomorphic 
to (M 8 U). 


This result brings our logic closer to the proposals in [5, 16] wherein 
only one atom is simultaneously assigned a value. The relation to other 
proposals will be discussed in Section 5. 


4 Card game actions 


In this section we apply the logic to model multi-agent system dynamics 
in the general settings of various game actions for card players, such as 
showing, drawing, and swapping cards. The precise description of card 
game dynamics is a prerequisite to compute optimal strategies to play such 
games [13, 18]. Card deals are also frequently used as standard represen- 
tation for cryptographic bit exchange protocols [22, 34], where communica- 
tions/transmissions should from our current perspective be seen as public 
announcements and/or assignments. 

Consider a deck of two Wheat, two Flax, and two Rye cards (w, x, y). 
Wheat, Flax and Rye are called the commodities. Three players Anne, Bill, 
and Cath (a, b, and c) each draw two cards from the stack. Initially, given 
a deal of cards, it is common knowledge what the deck of cards is, that all 
players hold two cards, and that all players (only) know their own cards. 
For the card deal where Anne holds a Wheat and a Flax card, Bill a Wheat 
and a Rye card, and Cath a Flax and a Rye card, we write wz.wy.xy, and 
so on. As the cards in one’s hand are unordered, wx.wy.xy is the same deal 
of cards as cw.wy.xy, but for improved readability we will always list cards 
in a hand in alphabetical order. There are certain game actions that result 


106 H. van Ditmarsch, B. Kooi 


WL.WY.LY — a —wea.ry.wy wy.ww sy = @ —wy.wy.wy 
c b wr.wyj ry z b 
IY. WE.LY Co WY.LYWI— >, et yy-wy-wa 
\ S / fot ae 
b c b c wy-Wx.ry i. 
NY a 


ry.wxe.wy — a —xry.wy.we 


FIGURE 2. On the left is the game state after the cards have been dealt and 
Anne received Wheat and Flax, Bill received Wheat and Rye, and Cath received 
Flax and Rye. On the right is part of the game state that results if Anne trades 
her Wheat card for Bill’s Rye card: only states resulting from trading Wheat for 
Wheat, and (what really happened) Wheat for Rye, are present. The actual deal 
of cards is underlined. In the figures, assume reflexivity and transitivity of access. 
The dotted lines suggest that some states are indistinguishable for Cath from yet 
other states but not present in the picture. 


in players exchanging cards. This is called trading of the corresponding 
commodities. Players attempt to get two cards of the same suit. That is 
called establishing a corner in a commodity. Subject to game rules that are 
non-essential for our exposition, the first player to declare a corner in any 
commodity, wins the game. For example, given deal wz.wy.xy, after Anne 
swaps her Wheat card for Bill’s Rye card, Bill achieves a corner in Wheat, 
and wins. Of course, players can already achieve a corner when the cards 
are dealt. This six-card scenario is a simplification of the ‘Pit’ card game 
that simulates the trading pit of a stock exchange [31, 13, 15]; the full game 
consists of 74 cards: 8 commodities of each 9 cards, and two special cards. 

An initial game state wherein players only know their own cards and 
nobody has won the game yet, can be modelled as an epistemic state. There 
are six such card deals. Assume that the actual deal is wx.wy.xy. The 
(hexagonally shaped) epistemic state (Pit, wa.wy.xy) in Figure 2 pictures 
what players know about each other. All six card deals have to occur to 
describe the epistemic information present, e.g., Anne cannot distinguish 
actual deal wa.wy.xy from deal wa.ry.wy, the other deal wherein Anne 
holds Wheat and Flax. But if the deal had been wa.ry.wy, Bill could not 
have distinguished that deal from wy.xy.wx, wherein Anne holds Wheat and 
Rye. Therefore, Anne considers it possible that Bill considers it possible 
that she holds Rye, even though this is actually not the case. 

The event wherein Anne and Bill swap one of their cards involves both 
epistemic and ontic change. Assume that, given deal wa.wy.xy, 
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Anne swaps her Wheat card for Bill’s Rye card. 


This informal description is not specific enough to be modelled as an update. 

In the first place, the role of Cath in the event is not specified. The event 
where Cath is unaware of the swap taking place, is different from the event 
where Cath observes the swap and where all agents are commonly aware 
of this. If Cath had been unaware of the event, she would be mistaken 
about the actual state of the world. For example, she would incorrectly still 
believe that neither Anne nor Bill has a corner in a commodity, whereas 
Bill holds two Wheat cards after the trade (and we assume that he has not 
yet so declared). It is hard to conceive of such a scenario as a game: even 
in imperfect information games, such as Pit, a basic condition of fairness 
must be fulfilled for players to be able to act rationally. This means that 
all events should at least be partially observable, and that ‘what actually 
happened’ should be considered a possibility for all players. We therefore 
assume, for now, that Cath learns that Anne and Bill swap one of their 
cards, but not which card it is. (The ‘private swap’ will be modelled later, 
as another example.) 

Anne and Bill’s roles in the event are also underspecified. Anne may 
knowingly choose a card to hand to Bill (the obvious interpretation), or 
blindly draw one of her cards to hand to Bill. The latter is not obvious, 
given this description, but becomes more so if we see it as Bill drawing 
(therefore blindly) one of Anne’s cards. For now, assume the obvious. An- 
other specification issue is that we may think of Bill as receiving Anne’s 
Wheat card facedown and only then, in a subsequent action, picking it up. 
From our modelling perspective, Bill already can be said to own the card 
after he has been handed it, but before he has picked it up he does not yet 
know that he owns it. We first assume that players immediately ‘see’ the 
card they are being traded (in this case maybe not the most obvious choice, 
but the simplest one to model). In other words: Anne and Bill jointly learn 
the new ownership of both cards. 

To describe this multi-agent system and its dynamics, assume a propo- 
sitional language for three agents a,b,c and with atoms u? expressing that 
Anne holds n cards of suit u. For example, w2 expresses that Anne holds 
two Wheat cards. In the event where Anne and Bill swap Wheat for Rye, 
Anne gets one less Wheat card, Bill gets one more Wheat card, Bill gets 
one less Rye card, and Anne gets one more Rye card. In the update model 
we have to distinguish separate events for each card deal wherein this swap 
can take place, i.e., corresponding to wz.wy.xy, wx.cy.wy, and WY.LY.WT 
(in general this depends on a feature of the local states of the card swapping 
agents only, namely for both agents on the number of Wheat and Rye cards 
in their hands, in this specific case that information is sufficient to determine 
the entire card deal). In case the card deal is wa.wy.xy the precondition 
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and postcondition are 


If (we Ay? A wy A yp), then 
wł := L and w? := T and Wh := L and we ae 
and y$ := L and y4 := T and yj := L and yp := T. 


We name this event swap “”®Y(w,y). If two cards of the same suit 


are swapped, a simpler description is sufficient. For example, the event 
wherein Anne and Bill swap Wheat given deal wx.wy.xy is described as 
swap), ”"¥(w,w) with (the same) precondition and (empty) postcondi- 
tion 

If (wy Ay? A wy A yg), then Ø. 

From the point of view of an actual card deal, there are always four 
different ways to exchange a single card, namely for each agent either the 
one or the other card. All of these are clearly different for Anne and Bill, 
because they either give or receive a different card (we assumed that they 
know which card they give and see which card they receive). None of these 
are different for Cath. For different card deals, card swapping events are 
indistinguishable if those deals were indistinguishable. For example, the 
event where (Anne and Bill swap Wheat and Rye given wa.wy.xy) is in- 
distinguishable for Anne from the event where (Anne and Bill swap Wheat 
and Rye given wa.xy.wy), because card deals wa.wy.xy and wa.xy.wy are 
the same for Anne. 

Therefore, the update model for Anne swapping her Wheat card for 
Bill’s Rye card consists of 24 events. The preconditions and postconditions 
of the events are as above. The accessibility relations are defined as, for 
deals d,d’ € dom(Pit) = {wa.wy.zy, wx.cy.wy,...} and cards q,q’,q1,q4 € 
{w,z,y}, and accessibility relations R(a),R(b),R(c) in the epistemic 
model Pit: 


(swap?,(q, q’), swap, (q1, 9) € R(a) iff (d,d') € R(a),q=q and q = q 
(swap?,(q, q’), swap%,(q1,q)) € R) iff (d,d’) € R@b), q = qı and d'= q 
(swap4,(q, q'), swap% (q1, q4)) € R(o) iff (d, d') € R(c) 


We name the update model Swap. The event of Anne and Bill swapping 
Wheat for Rye has therefore been modelled as update (Swap, swap% (w, y)). 
The result of executing this update model in epistemic state (Pit, wx.wy.xy) 
has the same structure as the update model, as the preconditions are unique 
for a given state, and as access between events in the update model copies 
that in the epistemic state. It has been partially visualized in Figure 2. 
An intuitive way to see the update and the resulting structure in Fig- 
ure 2, is as a restricted product of the Pit model and nine card swapping 
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events swap(q, q1), namely for each combination of the three different cards. 
Figure 2 then shows just two of those copies, namely for swap(w,y) and 
swap(w, w). For example, the event swap(w, y) ‘stands for’ the three events 
swapu, "8 (w, y), swap (w, y), and swapiy 949 (w, y). 

Why did we not define such swap(q, q1) as updates in their own right, in 
the first place? Although intuitive, this is not supported by our modelling 
language. We would like to say that the postconditions are ‘Anne gets 
one less Wheat card, and Bill gets one more Wheat card,’ and similarly 
for Rye. But instead, we only can demand that in case Bill already had a 
Wheat card (extra precondition), then he now has two, etc. Incidentally, we 
can also add non-deterministic choice to the update language by notational 
abbreviation, as [a U Bly © (faly A [G]y) (this corresponds to taking the 
union of the epistemic state transformations induced by œ and 8). We 
can then define, in the update language, swap(w, y) = swapiy "7Y (w, y) U 

WY- LYWE wE.LY.WY 
swap, (w, y) Uswap,, (w, y). 

The case where Anne does not choose her card but Bill blindly draws 
one of Anne’s can also be modelled as an update. The accessibility for Anne 
then expresses that she is unaware which of her cards has been drawn: 


(swap4,(q, q’), swap%,(qi.q,)) E€ R(a) iff (d,d) € R(a) and g = q, 


This is somewhat counterintuitive when we still suppose that Anne observes 
which card she receives from Bill. (We’d have to imagine Bill blindly draw- 
ing one of Anne’s cards, Anne putting her remaining card facedown on the 
table, and receiving the card Bill gives her faceup.) A more realistic setting 
is then that Bill draws one of Anne’s card and ‘pushes the card he gives 
to Anne facedown towards her’. At that stage Anne can already be said 
to own the card, but not yet to know that. All four swapping actions for 
a given deal are indistinguishable for Anne (as they were and still are for 
Cath). 

Yet another event is where Anne chooses a card to show to Bill, and 
receives Bill’s card facedown (before she picks it up). Access is now 


(swap, (q, q'),swapd,(q1:41)) E R(a) if (d,d’) € R(a) and q= q 


Obviously, all these variables can be applied to Bill as well. 


Picking up a card. The action of picking up a card after it has been 
handed to you, has another description (see, using another dialect of the 
language, [15]). One aspect is interesting to observe in the current context. 
Imagine that given deal wx.wy.xy after Anne and Bill swapping Wheat and 
Rye, Bill receives the card facedown in the following way: after having laid 
down his remaining card (a Wheat card) facedown on the table, Anne puts 
her Wheat card facedown on top of it or under it in a way that Bill cannot 
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see that. He now picks up his two cards. How does Bill know which of 
the two Wheat cards that he then holds, is the received card? He does not 
know, but he does not care either. By looking at his cards after the swap, 
he effectively learns that he holds two Wheat cards (which was already true 
after having received the card), and after that event he then knows that he 
holds two Wheat cards. A neat way to express that he learns the suit of the 
card he received, is to say that there is a suit for which he learns to have 
one more card than he knows. This makes sense in the general Pit game 
setting wherein one only is allowed to trade a certain number of cards of 
the same suit. This is formalized in our setting by an update (U,e) with an 
event with precondition pre(e) = w \—[b]w? (and empty postcondition), as 
a result of which Bill then knows to hold two Wheat cards, i.e., [U, e][b]w?. 
Private swap. The event where Anne and Bill swap Wheat for Rye but 
where Cath is unaware of the event is modelled by a four-event update with 
events swap "Y (w, y), swap "7T (w, y), swap, "9 (w, y) and skip, 
such that for Anne and Bill access among the swap events is as already dis- 
cussed (including all variations), but where ‘Cath thinks nothing happens’: 
for the deals d in the three swap events: (swap%,(w, y), skip) € R(c), and 
(skip, skip) € R(a), R(b), R(c). 


5 Comparison to other approaches 


Action model logic. Dynamic modal operators for ontic change, in ad- 
dition to similar operators for epistemic change, have been suggested in 
various recent publications. As far as we know it was first mentioned by 
Baltag, Moss, and Solecki as a possible extension to their action model logic 
(for epistemic change), in [5]. This was by example only and without a 
language or a logic. A precise quotation of all these authors say on ontic 
change may be in order: 


Our second extension concerns the move from actions as we have been 
working them to actions which change the truth values of atomic sen- 
tences. If we make this move, then the axiom of Atomic Permanence” 
is no longer sound. However, it is easy to formulate the relevant ax- 
ioms. For example, if we have an action a which effects the change 
p := pAn7gq, then we would take an axiom |a]p => (PRE(a) > pA7q). 
Having made these changes, all the rest of the work we have done goes 
through. In this way, we get a completeness theorem for this logic. [5, 
p. 24] 


The logic that we present here is a realization of their proposal, and we 
can confidently confirm that the authors were correct in observing that 
“all the rest (...) goes through”. To obtain such theoretical results the 


5 Le., [a]p > (PRE(a) > p), [5, p. 15]. 
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notion of simultaneous postconditions (assignments) for a finite subset of 
atomic propositional letters is essential; this feature is not present in [5] 
(but introduced in [20]). 

In a later proposal by Baltag [2] a fact changing action flip P is proposed 
that changes (‘flips’) the truth value of an atom P, with accompanying 
axioms (for the proper correspondent action a resembling a single-pointed 
action model) [a]p > (pre(a) — ~p) if “p changes value (flips) in a”, and 
otherwise [a]p > (pre(a) — p) [2, p. 29]. The approach is restricted to ontic 
change where the truth value of atoms flips. In the concluding section of 
[2], the author defers the relation of this proposal to a general logic of ontic 
and epistemic change to the future. 


Recent work in dynamic epistemics. More recently, in a Multi-Agent 
Systems application-driven line of research [16, 15] assignments are added 
to the relational action language of [12] but without providing an axiomati- 
zation. In this setting only change of knowledge is modelled and not change 
of belief, i.e., such actions describe transformation of $5 models only, such 
as Kripke models corresponding to interpreted systems. 

A line of research culminating in Logics of communication and change 
[10] also combines epistemic and ontic change. It provides a more expressive 
setting for logical dynamics than our approach. The logic presented here 
is a sublogic of LCC. In [10] the focus is on obtaining completeness via 
so-called reduction axioms for dynamic epistemic logics, by extending the 
basic modal system to PDL. Our treatment of postconditions, also called 
substitutions, stems from [20]. In the current paper we focus on specific 
semantic results, and, as said, we use a dynamic epistemic ‘dialect’, not full 
PDL. 

A recent contribution on combining public ontic and epistemic change, 
including detailed expressivity results for different combinations of static 
and dynamic modalities, is found in [28]. Our work uses a similar approach 
to ontic events but describes more complex than public events: the full 
generality of arbitrarily complex events involves exchange of cards among 
subgroups of the public, and other events with a ‘private’ (as opposed to 
public) character. 

Finally, a general dynamic modal logic is presented in [33], where ontic 
changes are also studied. The semantics of this logic uses tree-like struc- 
tures, and fixed points are introduced in the language to be able to reason 
about updates. 


Belief revision. An independent recent line of investigation combining 
epistemic with ontic change arises from the belief revision community. Mod- 
elling belief revision, i.e., epistemic change, by dynamic operators is an old 
idea going back to Van Benthem [7]. In fact, this is one of the two original 
publications—together with [32]—that starts the area of dynamic epistemic 
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logic. For an overview of such matters see [3, 14, 17]. But modelling on- 
tic change—known as belief update [27|—in a similar, dynamic modal, way, 
including its interaction with epistemic change, is only a recent focus of on- 
going research by Herzig and collaborators and other researchers based at 
the Institut de Recherche en Informatique de Toulouse (IRIT) [25, 26, 29]. 
Their work sees the execution of an event as so-called progression of infor- 
mation, and reasoning from a final information state to a sequence of events 
realizing it as regression—the last obviously relates to planning. The focus 
of progression and regression is the change of the theory describing the in- 
formation state, i.e., the set of all true, or believed, formulas. As already 
mentioned in Section 3, the results for arbitrary belief change in Proposi- 
tion 3.2 and following corollaries can potentially be applied to model belief 
update in the AGM tradition. 


Interpreted systems. In a way, dynamic epistemic logics that combine 
epistemic and ontic change reinvent results already obtained in the inter- 
preted systems community by way of knowledge-based programs [30, 21]: 
in that setting, ontic and epistemic change are integrated. Let us point out 
some correspondences and differences, using the setting of van der Mey- 
den’s [30]. This work investigates the implementation of knowledge-based 
programs. The transition induced by an update between epistemic states, 
in our approach, corresponds exactly to a step in a run in an interpreted 
system that is the implementation of a knowledge-based program; the rela- 
tion between both is explicit in van der Meyden’s notion of the progression 
structure. Now the dynamic epistemic approach is both more general and 
more restrictive than the interpreted systems approach. It is more restric- 
tive because dynamic epistemics assumes perfect recall and synchronicity. 
This assumption is implicit: it is merely a consequence of taking a state 
transition induced by an update as primitive. But the dynamic epistemic 
approach is also somewhat more general: it does not assume that accessibil- 
ity relations for agents are equivalence relations, as in interpreted systems. 
In other words, it can also be used to model other epistemic notions than 
knowledge, such as introspective belief and even weaker notions. 
Knowledge-based programs consist of joint actions (a¢,@1,...,@n) where 
Qe is an action of the environment and where a1,...,an are simultaneous 
actions by the agents 1 to n. An agent a acts according to conditions of 
the form ‘if y’ do a’, if p” doa", ...’ etc. Let us overlook the aspect that 
conditions y’ have the form of known formulas (by agent a). Still, such 
statements look familiar to our alternative format for what goes on in an 
event, as in ‘for event e: if p, then pı := Ui, ..., and pn := Wn.’ (see 
after Definition 2.4 on page 91). This correspondence is not really there, 
but the similar format is still revealing. The different cases in a knowledge- 
based program are like the different events in an update model, and they 
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equally express non-determinism. This is a correspondence. There are also 
differences. In dynamic epistemics, the condition y in ‘if y, then pı := Y1, 

.’ is both an executability precondition and an observation. Inasmuch 
as it is an observation, it determines agent knowledge. In the interpreted 
systems approach, observations are (with of course reason) modelled as 
different from preconditions. The assignments such as pı := %1 in the ‘then’ 
part of event descriptions are merely the ontic part of that event, with the 
‘if’ part describing the epistemic part. Epistemic and ontic features together 
correspond to actions such as a’ in ‘if y’ do a”. In the interpreted systems 
approach, epistemic and ontic features of information change are therefore 
not separately modelled, as in our approach. 


6 Further research 


An unresolved issue is whether updates can be described as compositions of 
purely epistemic events (preconditions only) and purely ontic events (post- 
conditions only). In [25] it is shown for public events, for a different logical 
(more PDL-like) setting. Such a result would be in the line of our other 
normalization results, and facilitate comparison to related approaches. The 
result in Section 3.1 seems to suggest this is possible, since a transition from 
one epistemic model to another is achieved by an epistemic event followed 
by an ontic event. However, the method described in Section 3.1 is geared 
towards the original epistemic model. We would like a decomposition that 
is based solely on the update, which would work regardless of the particular 
epistemic model to which it is applied. 

The logic can be applied to describe cryptographic bit exchange proto- 
cols, including protocols where keys change hands or are being sent between 
agents. The logic is very suitable for the description of protocols for com- 
putationally unlimited agents, such as described in the cited [22, 34]. Using 
dynamic logics may be an advantage given the availability of model check- 
ing tools for such logics, as e.g., the very versatile epistemic model checker 
DEMO [19] by van Eijck. The current version of DEMO only allows epis- 
temic events. But van Eijck and collaborators are in the process of extending 
DEMO with assignments (postconditions), needed to model events that are 
both epistemic and ontic. 

Our more fine-grained analysis of events may contribute to the descrip- 
tion and verification of more complex protocols that also include non-public 
events. An example that demonstrates the applicability of the logic to the 
analysis of such protocols is the (solution of the) ‘one hundred prisoners and 
a light bulb’ riddle (see e.g., [35]), of which we have a detailed analysis in 
preparation that we consider of independent interest. 

The results for ‘arbitrary belief change’ suggest yet another possibly 
promising direction. Under certain conditions arbitrary formulas are real- 
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izable. What formulas are still realizable if one restricts the events to those 
considered suitable for specific problem areas, such as forms of multi-agent 
planning? And given a desirable formula (a ‘postcondition’ in another sense 
of the word), what are the initial conditions such that a sequence of events 
realizes it? This is the relation to AI problems concerning regression as 
pointed out in the introductory section [25], and also to reasoning given 
specific protocols, such as always has been the emphasis for knowledge- 
based programs in the interpreted systems community [21], and as recently 
investigated in [11] in a dynamic epistemic context. 
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Abstract 


We extend our work that uses ATL to reason about social laws. In 
a system with social laws, every agent is supposed to refrain from 
performing certain forbidden actions. Rather than assuming that all 
agents abide to the law, we reason about what happens if certain 
agents do, i.e., they act socially, while others don’t. In particular, we 
are interested in the strategic abilities, under such mixed conditions. 
We also compare our approach with one in which labels are added to 
ATL that record whether agents behaved socially or not. 


1 Introduction 


A multiagent system [15] on the one hand aims at treating the agents as 
autonomous entities, whose behaviour should not be over-specified or too 
constrained, while at the same time one wants this system to achieve, or 
maintain certain objectives. As a consequence, one of the defining prob- 
lems in multiagent systems research is that of coordination—managing the 
interdependencies between the actions of multiple interacting agents [1, 15]. 
There are broadly two techniques to approach this. Online techniques aim 
to equip agents with the ability to dynamically coordinate their activities, 
for example by explicitly reasoning about coordination at run-time. In con- 
trast, offline techniques aim at developing a coordination regime at design- 
time, and hardwiring this regime into a system for use at run-time. There 
are arguments in favour of both approaches: the former is potentially more 
flexible, and may be more robust against unanticipated events, while the 
latter approach benefits from offline reasoning about coordination, thereby 
reducing the run-time decision-making burden on agents [15]. 

One prominent approach to “offline” coordination is the social laws 
paradigm, introduced largely through the work of Shoham, Tennenholtz, 
and Moses [12, 11, 13, 14]. A social law can be understood as a set of rules 
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imposed upon a multiagent system with the goal of ensuring that some de- 
sirable global behaviour will result. Social laws work by constraining the 
behaviour of the agents in the system—by forbidding agents from perform- 
ing certain actions in certain circumstances. Limiting the agents’ abilities 
as this may sound, this may also open new opportunities for them: assum- 
ing that other agents abide to the norms, may imply that an individual can 
in fact achieve more. Shoham and others investigated a number of issues 
surrounding the development and use of social laws, including the computa- 
tional complexity of their synthesis, and the possibility of the development 
of social laws or conventions by the agents within the system themselves. 

In [5], we extended this social laws framework. In particular, we argued 
that Alternating-time Temporal Logic (ATL) provides a rich and natural 
technical framework within which to investigate social laws and their prop- 
erties. In this framework, a social law consists of two parts: an objective 
of the law, written as an ATL specification, and a behavioural constraint, 
which is a function that for every action returns the set of states where 
that action is forbidden from being performed. The objective of the law 
represents what the society aims to achieve by adopting the law, and the 
behavioural constraint corresponds to the requirements that the law places 
on the members of society. In [4] we then added an epistemic flavour to this 
ATL-based approach to social laws, enabling one not only to express that 
an agent should behave in certain ways given his information, but also that 
certain information should emerge by following a social law. 

In this paper we extend our social laws framework further. In our pre- 
vious work [5, 4], it is assumed that once social laws are imposed on the 
system, all the agents will abide by these laws. However, this does not seem 
to be the most realistic way of modelling social laws. Certainly in human 
societies, just because laws are imposed does not mean that all members 
of the society will follow these laws. In this paper we do away with the 
need for this assumption and give agents the choice of whether to follow 
these laws or not. We make a distinction between agents acting physically 
and agents acting socially. Acting physically—we use this in lack of a bet- 
ter term we know—corresponds to performing any action that is physically 
possible to be performed, in the sense of being possible by the system de- 
scription. Acting socially then corresponds to performing any action from a 
subset of these physical actions, known as the social actions. Social actions 
are those that are consistent with the social laws imposed on the system. 
An example scenario is in the case of traffic laws. Traffic lights are used 
to coordinate the flow of traffic on our roads to ensure no collisions occur 
when cars cross each other’s paths. Cars are only allowed to move from the 
traffic lights when the light is green. Acting socially (and physically) would 
correspond to only moving when the lights are green. However, acting phys- 
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ically (but not socially) could correspond to moving when the lights are red. 
This would be an illegal action, but still a physically possible action. It is 
important to note that all social actions are also physically possible actions. 

Agents now have both physical and social strategies, where the set of 
social strategies is always a subset of the physical strategies. So after giving 
the agents these extra possibilities, we need to extend the logic to be able to 
reason about whether an agent or coalition of agents is able to act socially 
or physically (of course if an agent is able to act physically, this agent can 
choose to only perform social strategies if it desires). We introduce two 
logical languages: in Social ATL (SATL), one can express properties like the 
following: “Even if the coalition abides by the laws, and all the other agents 
neglect them, our coalition can achieve a particular temporal property”, 
and in Social ATEL (SATEL) one can also refer to informational attitudes of 
the agents, like in: “the server knows that, where all the clients are act- 
ing socially, then eventually each granted write-permission will terminate”. 
Finally, we investigate an alternative approach for expressing properties of 
systems that refer to whether the agents are acting socially or physically. 
Rather than using the logical language, Social ATL, we see to what extent 
we can capture the same notions using only ATL and ATL’. 

This paper is structured as follows: We first introduce the semantic 
structures that our model is based on. We call these Social Action-based 
Alternating (Epistemic) Transition Systems (SAAETS and SAATS). We then 
introduce our logical languages, Social ATEL and Social ATL, and give its 
semantics. In Section 4 we introduce a case study known as the Alternat- 
ing Bit Protocol and go on to investigate various interesting properties of 
this model. In Section 5 we try to capture similar properties in an alterna- 
tive framework and find direct equivalences between the two. Finally, we 
conclude in Section 6. 


2 Semantic Structures 


In this section we introduce the semantic structures our model is based 
upon. The structures we use are called Social Action-Based Alternating 
Epistemic Transition Systems (SAAETSs). Our structures are most similar 
to the AAETSs we introduced in [4]. However, our structures differ from those 
in several ways. Instead of having one action pre-condition function, p, we 
now have both a physical action precondition function, p, and a legal action 
precondition function, Z. Also, where the emphasis in [5] is on implementing 
a social law on a system, we do not consider such updates: rather, we assume 
the £ function is given, constraining the set of possible transitions, and we 
are not concerned about which social goal or objective such constraints are 
supposed to achieve. Rather, this framework is used to investigate social 
laws at the system level. Hence the system designer is able to see which 
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properties hold depending on who follows the social laws. 


Formally, an SAAETS is a (2n + 8)-tuple 


(Q, qo, Ag, AC1,. .., ACn, “1, ---; ~n, P, L, T,®, T) where 
Q is a finite, non-empty set of states; 
qo € Q is the designated initial state of the system; 
Ag = {1,...,n} is a finite, non-empty set of agents; 


Ac; is a finite, non-empty set of actions, for each i € Ag, where Ac; N 
Ac; = Ø for alli # j € Ag. With Acag we mean the set of all 
actions: Acag = Vieng Ac;. Moreover, Jag is the set of joint actions 
j = (@1,.--,@n), with action j; = a; in Ac; (i < n); 


~i C QxQ is an epistemic accessibility relation for each agent i € Ag. 
Each ~; must be an equivalence relation. 


p : Acag > 22 is a physical action precondition function, which for 
each action a € Acag defines the set of states p(a) from which a may 
be physically performed; and, 


L: Acag > 22 is a legal action precondition function, which for each 
action a E€ Acag defines the set of states L(a) from which a may be 
physically and legally performed. We require that for all a € ACag, 


L(a) C pla). 


T: QX Ing — Q is a partial system transition function, which de- 
fines the state T(q, j) that would result by the performance of j from 
state q—note that, as this function is partial, not all joint actions are 
possible in all states (cf. the physical action pre-condition function 
above). 


® is a finite, non-empty set of atomic propositions; and 


m : Q — 2° is an interpretation function, which gives the set of 
primitive propositions satisfied in each state: if p € a(q), then this 
means that the propositional variable p is satisfied (equivalently, true) 
in state q. 


As with AATSs, SAAETSs must satisfy two coherence constraints: Firstly, 
non-triviality [11]: Agents always have at least one legal action: Yq € 
Q,Vi € Ag,da € Ac; s.t. q E L(a). Secondly, consistency: The p and T 
functions agree on actions that may be performed: Vq,Vj € Jag, (q, j) € 
dom 7 iff Vi € Ag,q € p(j;). Sometimes we are not interested in knowledge 
and leave out the accessibility relations: such a system is called a SAATS. 
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Example 2.1 (The Train Example). We refer to this system as Train. 
There are two trains, one of which (E) is Eastbound, the other of which 
(W) is Westbound, each occupy their own circular track. At one point, 
both tracks pass through a narrow tunnel—a crash will occur if both trains 
are in the tunnel at the same time. 

We model each train i € Ag = {E,W} as an automaton that can be 
in one of three states: “away,” (the initial state of the train); “waiting,” 
(waiting to enter the tunnel); and “in,” (the train is in the tunnel). Initially, 
both trains are away. Each train i € {E,W} has two actions available. They 
can either move or idle. Idling causes no change in the train’s state. If a 
train 2 moves while it is away,, then it goes to a waiting, state; moving while 
waiting, causes a transition to an in; state; and finally, moving while in; 
causes a transition to away, as long as the other train was not in the tunnel, 
while if both trains are in the tunnel, then they have crashed, and will idle 
indefinitely. In order to prevent this from happening, train F is not allowed 
to move when either W is waiting or in the tunnel, while £ forbids W to 
move when Æ is in. See Figure 1, where ~; (q) is shorthand for {q | q ~i q'} 
(i€ {E,W}). So ~;(q) are those states that look similar to q, for agent i. 
In our example, each train knows about itself where it is: away, waiting or 
in the tunnel. 


Given an agent 7 € Ag and a state q E€ Q, we denote the physical options 
available to i in q by p-options(i,g) = {a | a € Ac; and q € p(a)} and, 
similarly, the legal options available to i in q by ¢-options(i,q) = {a |a € 
Ac; and q E€ &(a)}. Let Q* be the set of finite sequences of elements of 
Q, with typical element u,v € Q*, and where u -q denotes concatenation 
of an element u from Q* with a state q E€ Q. Qt C Q* collects all finite 
sequences of length at least 1. Now we can define a physical strategy and 
a legal strategy for an agent. A physical strategy for an agent i € Ag is a 
function: qy; : Qt — Ac; which must satisfy the constraint that y,;(u-q) € p- 
options(i,q) for all u € Q* and q E€ Q. A legal strategy for an agent i € Ag 
is a function: 6; : Qt — Ac; which must satisfy the legality constraint that 
di(u- q) € Loptions(i, q) for all u E€ Q* and q E€ Q. 

A physical strategy profile for a coalition G = {1,...,k} C Ag is a tuple 
of physical strategies (71,..., Yk), one for each agent i € G. Similarly, a 
legal strategy profile for a coalition G = {1,...,k} C Ag is a tuple of legal 
strategies (ô1,..., ôx), one for each agent i € G. By Tg, we denote the set of 
all physical strategy profiles for coalition G, whereas Ag C Tg denotes the 
set of all legal strategy profiles for G. By og, we denote a strategy profile for 
coalition G where we are not concerned about whether the strategy profile 
is legal or not; if øG € Tag and i € Ag, then we denote it’s component of og 
by ch. Given a coalition G, a grand coalition strategy profile cag can be 


considered as a tuple (oa, OG)» where og represents the choices made by 
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States and Initial States: 

Q = {q0, q1, q2, 93, 94,95, 96,97,98} Initial state qo 

Epistemic relations: 

~e (qo) = {q0, 91,42} ~e (q3) = {43;45;q6} ~e(qs) = {44, q7, q8} 
w (qo) = {40,9394} ~w (q1) ={%1,95,97} ~w (42) = {42; 96, q8} 

Agents, Actions, and Joint Actions: 

Ag = {E,W} Acg = {idlez, mover} Acw = {idlew,movew } 


Jag = {(idlez, idlew), (idlez, movey,), (moves, idlew }, (movez, movew )} 
SS Seer Oe cca 

Jo ji J2 j3 
Transitions/Physical action pre-conditions: 


—— TERRI 
go UJ=Jo ia ee A 

. . . . $: = 
~ Jn ifj=j T(q, j) = A 
T(qo,j) = Pees qo if j = j2 
q3 UJ = j2 fj $ 

. . . L ER 
qs ifj = j3 q s 

% é x 5 1 = 
q WIJ =Jo 7 me 2 

. . a . ) 1 = 
> Je ifj=j tlg, j) =4 115 
T(q, j) = Boat th q7 iff =je 
qs HJ = J2 fj , 

g é $ 1 = 
de HJ = 33 2 a X 

. . . y $ = 
q2 ifj = jo A a = 

. . . . £ 1 = 
~  Jgo iffy =f T(q6;5) = ene 
T(q2,3) = th oF, qs if j= j2 
do UJ = j2 fj P 
. . . 1 = c 
q3 ifj = j3 a a 2 

. . £: 1 = 
q3 ifj = jo 1 ve A 

è $ f 1 = 
, sos isn rand By 
T(q3,3) = meats qı iff = j2 
qd -u7 =] ifi $ 
DETA ; q2 uJ =3J3 

q7 ifj = j 

1533 tlas) =a (k =0...3) 


Legal action precondition function: 
L(idleg) Q (movez) = Q\{q5,¢6} 
L(lidew) = Q &(movew) = Q\{q7} 
Propositional Variables: 
® = {away p, away y , waiting p, waitingw, ing, inw } 


Interpretation Function: 


= ™(q4) {ing, away y } 
™(qo) = {away z, awayw } n(qs) = {waiting ,, waitingy,} 
mq) = {away p, waiting yy } ™(qe) = {waiting,, inw} 
mq) = {away p, inw } m(q7) = {ing, waiting,y,} 
w(q3) = {waiting,, away, } m(qs) = {ing,inw} 

FIGURE 1. The SAAETS for the trains scenario. 


Social Laws and Anti-Social Behaviour 125 


agents in G and 9% represents the choices made by all other agents in the 
system. 

Given a strategy profile cag and non-empty sequence u:q € QT, let 
out(Cag, H q) denote the next state that will result by the members of 
Ag acting as defined by their components of oa, for one step from yp - q. 
Formally, out(cag, u: q) = T(q,j) = q', where Trg (HE -q) = ji for i © Ag. 
Given a strategy profile cag for the grand coalition Ag, and a state q € Q, 
we define comp(cag,q) to be the run that will occur if every agent i € Ag 
follows the corresponding strategy o;, starting when the system is in state 
q E Q. Formally, comp(cag,g) = A = A[OJA[1]... where A[0] = q and 
Vu EN: Afu + 1] = out (oag, A[OJA[L] . . . Afu]). 

Given an SAAETS, S, and the initial state of the system, go, we define a 
set of socially reachable states as follows: 


sreach( S) = {q | ddag E Aag and Ju € N s.t. q = comp(dag, go) [wu] } 


The socially reachable states are all the states which are reachable when 
every agent in the system performs only social strategies. 


3 Social ATEL 


In this section we define the logical language used to express social laws 
in our framework. Our language is essentially an extension of ATEL [7]. In 
SATEL, we reason about physical and about social strategies. Our modalities 
are of the form ((G))¥, where x and y denote which kind of strategies G and 
Ag \ G, respectively, are allowed to use: only social strategies (denoted by 
s), or all their available ones (denoted by p). For example, the formula 
(G)) goal expresses that there exists a strategy for the coalition G, such 
that, no matter what the other agents do as long as they only follow social 
strategies, at some point in the future some goal state will occur. We can 
also require all the agents to act socially, e.g., KG}; [J—fail, which expresses 
that G has a social strategy, such that, no matter what the other agents 
do, providing they act socially, the system will never enter a fail state. 
Finally, consider the nested formula, (GYO KG} Ly, which reads: “G 
can ensure, by using a social strategy, and assuming that all the others also 
act socially, that in the next state, G can ensure, again by acting socially, 
that even if the others from now on act non-socially, y will always hold”. 
Or, a bit more informally: “if we require G to act socially, and the others 
socially for at least the first step, but unconstrained thereafter, then G can 
guarantee that always y”. 

ATEL adds knowledge operators K; on top of ATL. However, on top of 
that we add operators to express more enhanced informational attitudes. 
First of all, now that the possibility of violating social laws exists, we define 
a notion of social belief, i.e., belief under the assumption that all agents 
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in the system are acting as they should do according to the social laws 
imposed. We introduce a belief operator SB;, where S Biọ expresses that, 
if 2 assumes that everybody acts socially, he believes y to be the case. For 
example, if we use ọ to denote the fact that a car is stopped at a red traffic 
light, S.B;y means that agent i believes that a car is stopped at a red traffic 
light, under the assumption that all agents in the system are acting socially. 
So in all indistinguishable social states to agent i, a car is stopped at a red 
traffic light. An obvious difference with the standard notion of knowledge 
is that social belief is not veridical: it is perfectly well possible that agent i 
socially believes y (SBi) while ~g at the same time. 

Formally, the set of formulae, formed with respect to a set of agents Ag, 
and a set of primitive propositions ®, is given by the following grammar: 


y == ply yV y | Ky | SBi | (C) OY | (G)% Oe | (G) pU p 


where p € ® is a propositional variable, G C Ag is a set of agents, i € Ag is 
an agent, and x and y can be either p or s. 

We now give the truth definition of Social ATEL formulae on an SAAETS 
S and a state q. For G C Ag and z € {s, p}, let Strat(G, x) = TG if x = p, 
and Strat(G, £) = Ag if x = s. 


S, q= p if p€ (gq) (where p E ®) 
S, q E 79 iff S, q FY; 
SqFevy if S,q- yor Sq HE Y; 


S, qH KG) Op iff dog € Strat(G, x) s.t. Yog € Strat(G,y), if 
e r a have aE 


Sq H (G)% Oe iff dog € Strat(G,x) s.t. Yog € Strat(G,y), 
if A = comp((og,og),q), we have for alln € 


N, S,A[n] E ¢; 
S, qH KG) pU iff Jog € Strat(G,x) s.t. Yog € Strat(G,y), 
if A = comp((oc,o@),q), there is k € 


N s.t. S, à [k] =| w, and for all n with 0 < n < k, 
we have S, A [n] = y; 


S,q E Kip iff for all q’ such that q ~; g : S, d F 9; 
S, q H| S Bip iff for all q € sreach(S) such that q ~i q’, we 
have S, q! = y. 


” a 37 


Other connectives (“A”, “>”, “«>”) are assumed to be defined as ab- 
breviations in terms of ~, V. ((G))%y is shorthand for (G))¥TUy. We 
write (z)) rather than ({7})). Validity of y, written = y is defined as usual. 
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An objective formula is a purely propositional formula, with no knowl- 
edge, coalitional or temporal operators. The sublanguage obtained from 
SATEL by leaving out the epistemic and social belief operators is SATL. The 
language of ATL can be seen as being subsumed by SATL. Coalitional modal- 
ities in ATL are implicitly indexed with a p: the only type of strategy in ATL 
is physical. In ATL, one writes ((G)) Oy (corresponding to our ((G))? Oy), 
and similarly for the other temporal operators. Note that in SATEL and ATL 
every temporal operator is immediately proceeded by a coalition modal- 
ity. If one drops this constraint on ATL, the thus obtained language is 
called aTL*. Examples of ATL* formulas are ((G))(T(y —> Ow) (‘G can 
cooperate such that whenever y occurs, eventually ~ will happen’) and 
(G)(OeV OCO¢v OOO») (G has a strategy that ensures that within 
three steps, y’). 

In ATL, the cooperation modality (())T (where T is a temporal formula, 
i.e., a formula of which the main operator is either O, (| or U/) denotes 
a special case: it means that the empty set of agents has a strategy, such 
that, no matter what the other (i.e., all) agents do, T holds. In other 
words, no matter what the agents in Ag do, T. This resembles the CTL 
operator AT: on every future path, T. Similarly, (Ag))T means that the 
grand coalition has a strategy such that, no matter what the empty coalition 
does, T. In CTL terminology: ET, or, for some path, T. For SATEL this gives 
us the following. Since in (())%, the constraint to play an x-type of strategy 
is a constraint for nobody, it does not really matter whether x is s or p. 
Similarly, in ((Ag))’T' the constraint to play a y-type of strategy is a void 
restriction for Ag \ Ag, i.e., the empty set, so it does not matter whether x 
equals s or equals p. Summarising, we have 


OST = (pT and (Ag) T = (Ag), T 


As a convention, when having an empty coalition, we will only write ((} 
and (Op which is no restriction, given the equivalence above. Similarly, for 
the full coalition, we will only write (Ag); and (Ag}p- 

Multiple ((G))% O operators are used as an abbreviation in the following 
way: 

nA G) Op ifn=1 
((G)2O) p= { ae OGY” O)"~*y otherwise 


x 


Social strategies Ag are a subset of the physical strategies Tg, and 
hence: 


x 


= (GHT > (G),T and H (G) T > (CJT (3.1) 


where T here is an arbitrary temporal formula and x and y are vari- 
ables over p and s. These properties express the following. The first, 
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(G)%T — (G)%T says that if a coalition G are able to enforce a prop- 
erty y by adopting social strategies, then they can also enforce this same 
property when adopting physical strategies (‘if you can enforce it nicely, 
you can enforce it anyhow’); and (G))?T — (G) ÀT can be interpreted as 
saying that if a coalition G are able to enforce a property y when playing 
against an adversary who is able to use physical strategies, then they can 
also enforce this property when playing against the same adversary when 
this adversary is constrained to use only social strategies (‘if you can beat 
an opponent when he can cheat, you can beat him when he plays by the 
rules’). 


4 Case Study 


We now present a case study in order to demonstrate Social ATEL. The case 
study is known as “The Alternating Bit Protocol” and is adapted from [6]. 
In this scenario there are three agents, a sender S and a receiver R, who 
wish to communicate through a communication environment, represented 
as an agent Æ. The sender S owns a tape X = (%,21,...), where each 
element x; comes from a given alphabet, say Alph = {0,1}. The goal of the 
protocol is that the contents of this tape are sent to R, who writes what he 
receives on a tape Y that, after k elements of X have been received, will be 
Y = (yo, y1---,Yk). The sender only sends one item x; at the time. The 
environment E determines whether messages are delivered or get lost: Æ 
does not alter the content of any message. And, although the environment 
is unreliable, it satisfies the fairness property that it will not loose messages 
forever: if an agent repeatedly sends the same message, eventually it will be 
delivered. We wish to design a protocol that satisfies the safety requirement 
that the receiver never prints incorrect bits, and the liveness requirement 
that every bit will eventually be written by the receiver onto Y. 

The obvious solution to this problem is to use acknowledgements to let 
the sender know an element x; has been received. So the sender would 
repeatedly send the current element x; until eventually it receives an ac- 
knowledgement back from the receiver, at which point it would go on to 
send the next element x;,;. The problem arises when the value of an item 
x;41 is the same as x;. The receiver does not know whether its acknowledge- 
ment for x; has been received, so the receiver does not know whether the 
message he currently receives (with the value of x;,1) is a repetition of x;, or 
whether this bit is supposed to represent x;+ on the tape. To overcome this 
problem, the sender can put more information on which element he sends 
by adding a “colour” to it: a 0 to every x; for even i, and a 1 for every odd i. 
So the alphabet will now be Alph’ = {x.0 | z € Alph}U{a.1 | x € Alph}. We 
also need two acknowledgements: ackO and ack1. Now the protocol works 
as follows: S first sends 79.0. When it receives ack0, it goes on to the next 
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state on the tape and sends 2,.1. R can see that this is a new message 
(since it has a different colour), and sends ack1 to acknowledge the receipt 
of it. S can also see that this is a different acknowledgement and starts to 
send 22.0, and so on. 

We model the scenario with an SAAETS called AltBit. We introduce 
variables with the following interpretation: SSX = 1 means that the last 
bit S sent was of type x.l; SRA = 1 indicates the last message received 
by S was ackl; RRX = 1 means the last bit R received was of type 2.1; 
and RSA = 1 indicates the last message R sent was ack1. In the object 
language, atom ssx will mean that SSX = 1, and -ssx denotes SSX = 0. 
Similarly for the other variables and corresponding atoms. 


A state in our system is defined to be a tuple 
qi = (SSX, SRA | RRX, RSA) 


where 


e SSX, SRA, RRX, RSA € {0,1} = B; and 


e qi, where 1 <i < 16, is the name of the state. This is just a decimal 
representation of the binary number the state corresponds to (e.g., 
q3 = (0,0 | 1,1)). The initial state of the system is q15 = (1,1 | 1,1): 
no information about the first bit xo is assumed to be sent or received. 


Given the nature of our states and the correspondence between variables 
and atomic propositions, we have: ssx € m((SSX,SRA | RRX, RSA) 
if SSX = 1, and similarly for the other variables. We are not inter- 
ested in knowledge of E, but for S and R we assume that they know 
the contents of their own variables: if q = (SSX, SRA | RRX, RSA) and 
q = (SSX', SRA’ | RRX', RSA’) are two states, then q ~s q' iff SSX = 
SSX' & SRA = SRA’, and similarly for ~p. 

In every state of the system, the sender has two physical actions available 
to it: send.0 and send.1, corresponding to sending a bit with colour 0 and 
sending a bit with colour 1, respectively. The receiver also has two physical 
actions available to it in every state of the system: sendack.0 and sendack.1, 
corresponding to sending an acknowledgement of a bit with colour 0 and 
colour 1, respectively. Actions of the environment are pairs (es, er), where 
eg is either dg or gg (i.e., either deny or grant the request of the sender), 
and er € {dpr, gr}. 

Concerning the transition function T : Q x Jag — Q, we have the follow- 
ing: 


r((SSX,SRA|RRX, RSA), (as,ap,az)) = (SSX', SRA’ | RRX’, RSA’) 
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where 
gsx' 1 if ag = send.1 
~ \0_ if ag = send.0 

SRA if ag =(-,dr) 

SRA’ =21 if ap = sendack.1 & ap = (-, gr) 
0 if ag = sendack.0 & ag = (-, gr) 
RRX if ap = (ds,:) 

RRX'=1 if ag = send.1 & ag = (gs,°) 
0 if ag = send.0 & ag = (gs,°) 


RSA! = 1 ip ar = sendack.1 
0 ifar = sendack.0 


This transition function reflects the fact that both S and R are free to 
decide which message they send: S can choose the value of SSX’ and R 
chooses that of RSA’. However, whether a message arrives depends on Æ 
as well. For instance, in order for a new state to have SRA’ = 1, saying 
that the last acknowledgment that S received was an ack1, either this value 
was just sent by R and this was granted by E, or S had received SRA’ = 1 
in a previous state, and in the meantime he did not receive any update. 

Regarding the legal transitions, of course, the idea is that S should 
alternate x.0’s, once known to be received, with z.1’s, and R is expected 
not to acknowledge receipt of a bit he did not receive. Let q be (SSX, SRA | 
RRX, RSA). Then: 


q€&(send.1) = SRA=0 q E€ &(sendack.1) = RRX=1 
q E€ €(send.0) = SRA=1 q € €(sendack.0) = RRX =0 


For E, rather than putting a legality constraint, we require a fairness 
constraint on its behaviour. We say that a computation À = qoq -+-+ is fair 
if 

e there are infinitely many indices n for which there is an action ag for 

the sender and an action ap for the receiver and some er € {gr, dr} 
such that qn+1 = T(qn, (as,ar,(gs,eR))). 


e there are infinitely many indices n for which there is an action ag for 
the sender and an action ap for the receiver and some eg € {gs, ds} 
such that Qn4+1 = T(n, (as, GR, (es, gr)))- 


In words: a fair computation is one in which both the request of the 
sender and that of the receiver are granted infinitely often. Although fairness 
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is a property of computations, we here can connect it to the legality of 
strategies of E. The legal strategies Ag of E are all strategies that guarantee 
that the resulting computation is fair, no matter what R and S do. 

A socially necessary fact y is a fact which should be true no matter 
what, providing all the agents in the system act in a social manner. A SNF 
y is defined as follows: 


SNF(¢) = ()); Oe 


4.1 Properties of the model 


Let us discuss some properties, and verify whether they hold, either in our 
system AltBit, or in AltBit,q15. Recall that (> Cly means that ọ is true 
no matter what the agents do, while (())? [y says that ~ is true, as long 
as everybody acts socially. For instance, under what circumstances is it the 
case that if S sends an item (say 2;.1), R will eventually receive it? For 
this, ‘only’ E has to act socially: we have 


(p Oss > (E); Orra) (4.1) 


This is good, and indeed, without demanding that F acts socially, he 
could indeed spoil it: we have (())? [)((sstAarra) > {E}; Orre), saying, 
that when ssz is true, E can guarantee, even if the others act socially, that 
this message will never arrive. Returning to (4.1), it is not hard to see 
that E’s next action is partially determined if he wants to ensure Orra: 
if E does not immediately grant the request of S (i.e., E plays (dg,dp) or 
(ds,gr)), and we allow S to act physically, it can from any next state on 
decide to send x.0 (where, socially speaking, it should send x.1 again), and 
then, no matter which strategy E chooses, rra may never be true again. In 
other words, we do not have 


Op Osse > ()O(rra v (Es Orre) (4.2) 


i.e., it is not the case that under all social strategies, in the next state the 
message has either been received, or else the environment can still guarantee 
it will be received eventually. (The counterexample is the earlier given 
strategies of E and S.) 

If E and S both are social, we have the following positive result, which 
says that if x.1 is sent, then E and S can socially guarantee that this 
remains true until it is received, and moreover, it will be received under any 
behaviour, as long as E and S act socially: 


())p (ssa > ((E, S))5(ssxU rre) A ~E, 5) Corre) (4.3) 


Now suppose R has received a message with odd parity: rra. How can 
we derive that S will know about this? In other words, for which group G 
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and attitudes x and y do we have AItBit, qi5 = (Op (rre > (G)! sra)? 
In case x = y = p it holds iff G D {R, E}: the receiver and a 
are needed to ensure that a received bit by the receiver is acknowledged to 
the sender. This only says that G can do it, however, and we also have 
(Op (rre —> (G) p O-sra), even for G = {R} and for G = {FE}: in 
both cases, G can ensure that S does not get acknowledged properly. For 
that, illegal actions are needed, however, and indeed we have (O (rr£ > 
a(R, EX) O-sra): If E and R act legally, it is impossible for them to not 
have a message from the sender ever acknowledged. Indeed, we even have 
(i (rr£U (rraArsa)): if everybody acts socially, received messages will be 
acknowledged and not overwritten in the meantime. 

Let us write SLT(y,w), (‘p socially leads to 7’) if the following holds: 
p > (O i(pU y). That is, if y is true, then, if everybody acts socially, it 
will stay true until some point where ~ is true. We then have: 


OO ( SLT((assx A asrax),(ssz A asrx)) A (4.4) 
SLT((ssxz A asra),(rra A rsx)) A (4.5) 
SLT ((rra A rsx), (ssx A srx)) A (4.6) 
(( (4.7) 
(( (4.8) 


SLT((ssx A srx), ({ssz A srx)) A 
SLT((>ssz A sra),(4ssx A asrx)) 


) 


The displayed formula describes some properties of a correct run: (4.4) 
ensures that when S has sent and received an even-coloured message and 
corresponding acknowledgement, he will next send an odd-coloured message. 
According to (4.5) this is (socially) guaranteed to lead to a state where R has 
received this odd-coloured message and where he also acknowledges receipt 
of it. Then (4.6) says that this will socially lead to a state where S was 
still sending this odd-coloured message, and has received the corresponding 
acknowledgement. This will lead then (4.7) to a state where S sends again 
an even-coloured message that is not yet acknowledged, after which (4.8) 
at some point, while S was still repeating the even-coloured message, this 
becomes acknowledged. 

Recall that a socially necessary fact y is an objective formula for which 
(O) Lg holds. They express that some unwanted states will never be 
reached. For instance, we have in our example that (ssrA sra) > (rraArsz) 
is a socially necessary fact: if everybody behaves, then if S is sending an 
even-coloured message which has been acknowledged, then R’s last received 
message must be even-coloured, and his last sent acknowledgement must be 
odd as well. 

To conclude this case study, let us look at some informational properties 
in the alternating bit protocol, in particular knowledge and social belief. If 
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the agents cannot assume others behave socially, the agents actually have 
no knowledge whatsoever about the other agent’s state. In other words, if 
agents can act arbitrarily, then very little knowledge ensues. For instance, 
in q15, S knows of course its own state, but has no clue of R’s: q15 = 
aK grra \aKsrrrx \aKgrsaA\ aK go7rsa. 

However, this is not the case for social beliefs, where the agents mu- 
tually assume to abide to the social laws. Recall that we have AltBit = 
SNF((ssa A sra) > (rra A rsx)). Hence, in q15, if S can assume that ev- 
erybody behaves socially, he is able to infer the state of R, i.e., we have 
AltBit,q15 H SBs(rra A rsa): the sender has the social belief about the 
correct contents of the receiver’s local state. 


5 Reducing Social ATL to ATL* 


In this section we introduce an alternative approach for expressing prop- 
erties of systems that refer to whether the agents are acting socially or 
physically. Rather than using our logical language, Social ATEL, introduced 
in Section 3, we see to what extent we can capture the same notions us- 
ing only ATL and ATL*. To this end, we introduce the notion of “good” 
and “bad” states, similar to the “red” and “green” states of Lomuscio and 
Sergot in [8]. This idea goes in fact back to a discussion in the deontic 
logic literature regarding the two notions of ought to do and ought to be 
(see for instance [10] and the references therein). Although we do not refer 
to actions in our object language, the framework discussed in the previous 
section can be seen to fit in the spirit of an ought to do approach: in the 
semantics, we specify which actions are forbidden in which states. In an 
ought to be based deontic logic, the emphasis is on which states should be 
avoided. A celebrated paper to link the two is [9], where the fact that ac- 
tion a is forbidden, is represented in a dynamic logic like language as [a]V, 
where V is a propositional atom flagging violation of a norm. In words: 
the action a is forbidden iff doing a leads to only states that mark that 
something bad has happened. 

The paper [8] puts this in an interpreted system context [2], where, 
rather than an atom V, they use a label green to signal green states, states 
that are reached without any violation. Of course the languages of [9], [8] 
and ours are all different: [9] uses a dynamic deontic logic, [8] interprets a 
deontic (epistemic) language over runs (structures with a temporal flavour) 
while we have both ability operators for different norms and temporal op- 
erators. A full comparison between the approaches would be an interesting 
exercise. Here, we will relate SATL to the languages ATL and ATL* that we 
enrich with a specific atom good (which plays the same role as ~V and green 
above). 
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More precisely, we modify our Social Action-Based Alternating Transi- 
tion Systems by introducing an atomic proposition for each agent, which is 
only true in the current state if the agent arrived here from the previous 
state using a legal action. So essentially, we are labeling the states based 
on how the agents arrived at each state. This gives rise to a larger state 
space in the modified systems, as now we have copies of each state, repre- 
senting all the combinations of the agents acting socially or physically to 
reach it. Ideally we would like to be able to reduce properties expressed 
in Social ATEL into ATL, expressed using these “good” atomic propositions, 
in order to automatically verify Social ATEL properties using existing ATL 
model checkers, such as MOCHA. However, since Social ATEL and ATL are 
interpreted over different structures, it is not feasible to find direct equiv- 
alences between the two. We can, however, express interesting properties 
using ATL and ATL*. 

Using the approach outlined above, we look at several types of Social 
ATEL properties and see how closely we can express these in ATL*. We in- 
vestigate the relationship between the two using three special cases of G, 
where G is the coalition of agents cooperating to achieve some objective. 
We look at the case where G is the grand coalition (Ag), the empty coali- 
tion (@), and finally, an arbitrary coalition (G). We show that there is a 
general pattern between Social ATEL properties and properties expressed in 
this approach, which holds regardless of the coalition type and the temporal 
operators being used. Finally, we prove equivalences between general for- 
mulae expressed in Social ATEL and formulae expressed using this approach, 
for each combination of the coalition acting socially or physically, while the 
other agents act socially or physically. 


5.1 Modifying SAATSs 


In this section we introduce the atomic propositions used to give a labeling 
to each state based on how each agent arrived there. This can either be by 
performing a legal action or by simply performing any physically possible 
action. We introduce an atomic proposition, gi, one for each agent i € Ag, 
with the interpretation that g; is true in the current state if agent 7’s last 
action was a legal one. This corresponds to agent i acting in a social manner 
(for one time step). GP = {g1,...,gn} is a set of good propositions where 
GP C ®. In order to reason about coalitions of agents, we introduce a 
proposition good(G) which holds if all the agents in G acted socially to 
reach the current state. good(G) is defined as follows: 


good(@) = N x 
icG 
Now we must modify SAAETSs with these g; propositions. It is impor- 
tant to note that the definition of the g; propositions comes from the ¢(a) 
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function in the given SAAETS. Now, given a SAAETS 
Sys = (Q, do; Ag, Aci, SS ACn, P; £ T, ®, T) 


in order to express properties of systems in this way, we need to convert the 
system into a modified system as below: 


Sys° = (Q°, q0, Ag, Aci, sie »ACn, p°, T°, B°, 7°) 
where the modified components have the following interpretation: 


e Q°: For each state q € Q, we now have at worst 2!48! copies of q 
to represent all the combinations of agents being “good” and “bad” 
(this is a worst case estimate, since we can leave out states that are 
not reachable from q8). We encode this extra information as gz, ,...,.,,; 
where x; € {0,1}. x; being 1 means agent it’s last action was a social 
one, whereas a 0 means it was anti-social. The new set of states 
formed, Q°, in the worst case, is of size |Q x 248|. See Figures 2 and 
3 for an example of how the train system is transformed; 

e q6: qo becomes qg, which is an abbreviation of gj, | , where Vi € 
Ag, x; = 1. This is the initial state of the system, which is the same 
as before, except g; is true for all agents; 


e p°: Vi € Ag, Va € Acag, Yq E Q, Va; € {0,1} : q € pla) € az, 
p° (a); 
e °: P is updated with the new g; propositions: ®° = ® U GP; 


shade 


e 7°: 
Vi € Ag, Vp € ®, Vq € Q, Yx; € {0,1}: 
p E T(q) > p E T° (qr, en) 


Vi € Ag, Vg; € GP : gi € T° (dey,...0,) © Ti = 


asig 


e r°: VIE Ag, Yq, q' € Q, Vai € Aci, Vti € {0,1} : I7(q, (ar,.. a) = 
q 7° (Gary ,...2m1 (1+ Ak)) = Tp (ary), fa lam? Where fi(vi) = 1 S 


qella)). 


So now we have modified SAAETSs to work with these g; propositions. 
Firstly, the set of states has been modified. We have a new copy of each 
state for all combinations of g;, for all agents. The initial state is the same 
as before, but g; is true for all agents, hence the system starts in a good 
state. The new action precondition function, p°, is directly equivalent to p 
for all states and actions, regardless of the g; propositions. In other words, 
if an agent can perform a in q, then the agent can perform a in qz,,... æn 
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no matter what the values of x; are. The set of atomic propositions, ®, is 
updated with the set of good propositions, GP. The truth definition func- 
tion, m, is the same as before for atomic propositions in ®. It is updated 
for the propositions in GP, where a g; proposition is true in a state where 
xi = 1. Finally, the transition function, 7, is updated in the following way: 
transitions are the same as before, but now, if agent i performs a legal ac- 
tion, in the resultant state, g; will be true and the x; subscript of the state 
will be 1. Performing an illegal action results in the g; proposition being 
false and the x; subscript of the state being 0. 


Remark 5.1. In this section we do not look at properties that refer to 
knowledge. We briefly consider how we would modify the epistemic acces- 
sibility relations from SAAETSs to account for these good propositions. We 
propose the following modification to the epistemic accessibility relations: 
~g: Vi € Ag, Yq, q' E€ Q, Yxi € {0,1} : q ~i d © Ge1,...,00 ~2 Ter, 


aniy 


Pakiy 


So if two states, q and q’ are indistinguishable in Sys, qx, æn and Qhi.. 2, 
will be indistinguishable in Sys°. This will now mean that agents will know 
if they have acted socially, and also if other agents have acted socially. This 
would allow us to formulate informational properties about systems where 
agents can reason about the behaviour of the other agents in the system. 
An alternative way to modify the epistemic accessibility relations would be 
as follows: 


‘eR: Vi € Ag, Yq, q' € Q, Niet, € {0, 1} 7d ™i q © dzxı En ~? drt. 


esis 


where x; = 2}. 

This is the same as above, but now agents only know whether they 
have acted socially themselves, and about how other agents have acted is 
known. This would mean agents would only be able to reason about their 
own behaviour. 


5.2 Reducing some formulae 


There are various interesting types of properties we can form with these 
good(G) propositions. We will construct example properties using the train 
system Train that we introduced in Example 2.1. Let Train? be the AATS 
after modifying Train. We can see the affect modifying the systems has by 
comparing Figure 2, showing the states and transitions of the standard train 
system, with Figure 3, which shows the same system after being modified. 
States are assumed to have a reflexive arc labelled ‘7,2’, which we omit for 
clarity. 


We start by looking at the following Social ATEL formula: 


Sys, q = KG}; Oe (5.1) 
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Key: ---- denotes an illegal transition 


FIGURE 2. State transitions in Train. The top atom in a state refers to 
train E, as does the top action at each transition. 


where ọ is assumed to be a propositional logic formula. This Social ATEL 
formula says that G has a social strategy, such that, no matter what the 
other agents do, providing they follow social strategies, y will always be 
true. We will now try to capture the above using these good(G) propositions 
where we take G to be the grand coalition, Ag: 


Sys°, der,...an F (Ag)) O (good(Ag) A p) (5.2) 


where Vi € Ag: x; = 1. This formula states that the grand coalition of 
agents has a strategy such that it will always be the case that good(Ag) is 
true and ¢ is true at the same time. This appears to express the same as 
(5.1) above. If we refer to the train scenario we can formulate the following 
example property: 


Sys°, qo, = (Ag)) O (good(Ag) A > (ing A inw)) (5.3) 
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Key: * denotes an illegal action 


FIGURE 3. State transitions in Train®. 


So this property states that the grand coalition should have a strategy so 
that it is always the case that the agents follow only social strategies and 
that both of the trains are not in the tunnel at the same time. Property 
(5.3) passes when we model check it in our implementation of Train®. 

Following on from what we said about acting socially leading to the 
goal, we can formulate the following, where now we take G to be the empty 
set, Ø: 


Sys°, der,...an F (()) ( O good(Ag) > Ll) (5.4) 


where Vi € Ag: x; = 1 and y is assumed to be a propositional logic formula. 
The above ATL* formula reads that on all paths, no matter what any of the 
agents do, good(Ag) always being true implies that y will always be true. 
In other words, if the agents always follow social strategies, the goal, y, 
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is achieved. Referring back to the train scenario, we look at this example 
property: 


Sys, qo, = (()) ( O good(Ag) > O= (ing Ainw)) (5.5) 


So this reads, no matter what the agents do, on all paths, if the agents 
always follow social strategies, this implies that the trains will never enter 
the tunnel at the same time. This sounds intuitively correct based on what 
we said earlier about social strategies leading to the objective of the social 
law being satisfied. Using Figure 3, we leave it to the reader to check 
that (5.5) is satisfied. 


Let us now consider 


Sys? , de,n F ()) (TJ good(Ag) > Ty) A (Ag)) C good(Ag) (5.6) 


where Vi € Ag : x; = 1 and ọ is assumed to be a propositional logic 
formula. So, as before in (5.4), this formula says that the agents always 
acting socially implies that the goal will always be achieved, but now also 
the grand coalition should have a strategy such that they will always act in 
accordance with the social laws. However, if we refer back to the definition 
of SAAETSs we see that there is a condition which states that agents should 
always have at least one legal action available to them in every state of the 
system. As Sys° is a modified version of Sys, this condition still holds in 
Sys°, thus guaranteeing that there is at least one social path, hence Ag will 
always have a strategy to ensure good(Ag) is true. This makes this property 
redundant, as it is directly equivalent to (5.4). 


Finally we consider the case where we have an arbitrary coalition, G: 


Sys°, qz,- zn F (G) ( O good(G) A ( O good(G) > Ly)) (5.7) 


where Vi € Ag: x; = 1 and y is assumed to be a propositional logic formula. 
So this formula reads that coalition G has a strategy so that agents in G are 
always good and that if the other agents in the system are always good then 
y will always hold. We needed to separate out the good(Ag) proposition 
into good(G) and good(G), as G has no control over what the agents in G 
do. Therefore, we can precisely capture (5.1) in this way. If we refer to the 


trains example, we can formulate a property such as the following: 


Sys°, qo, = (E) (LI good(£) A ( O good(W) > O> (ing Ainyw))) (5.8) 


This property states that the Eastbound train has a strategy so that it is 
always social, and if the Westbound train is always social, then the two 
trains will never be in the tunnel at the same time. This property holds 
and can be verified by inspection of Figure 3. 
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Now we consider another type of Social ATEL formula: 


Sys, q = (G) y (5.9) 


where ọ is assumed to be a propositional logic formula. This formula says 
that G has a social strategy, such that, no matter what the other agents do, 
as long as they all only follow social strategies, then y will be true either 
now or at some point in the future. We will now try to express a similar 
property using good(G) propositions, where we take G to be the empty set: 


Sys°, qzi,- en E ()) ( O good(Ag) > Ov) (5.10) 


where Vi € Ag: x; = 1 and y is assumed to be a propositional logic formula. 
This formula says that no matter what the agents do, on all paths, good(Ag) 
always being true means that y will either be true now or at some point 
in the future. That is to say that if all the agents always act socially, the 
goal y will eventually be achieved. 

We can also consider situations in which the other agents are constrained 
to social strategies and the coalition of interest can act in an unconstrained 
manner: 


Sys, q = (G)), De (5.11) 
where y is assumed to be a propositional logic formula. This formula states 
that G has a strategy to ensure y is always true, providing the other agents 
always act in a social manner. We can express something similar to this in 
the good states approach in the following way: 


Sys°, dei, en = (G)) (O good(G) > Oy) (5.12) 


where Vi € Ag: x; = 1 and y is assumed to be a propositional logic formula. 
So, here we are saying that G has a strategy such that no matter what the 
other agents in the system do, providing the other agents follow only social 
strategies, G can always achieve y. We can look at an example property of 
the same type as (5.12): 


Sys, dor. = (W)) ( O good(E) > O- (ing A^ inw)) (5.13) 


This property states that the westbound train has a strategy so that if 
the eastbound train always acts socially, then the trains will never enter 
the tunnel at the same time. This property holds and can be verified by 
inspection of Figure 3. 


5.3 Reducing Social ATEL to ATL* 


After investigating the above formulae, we have noticed a general pattern 
between formulae of Social ATEL and formulae expressed in this good states 
approach, which seems to follow regardless of the coalition type (grand, 
empty or arbitrary coalition) and the temporal operator being used. 
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5.3.1 Bisimulations between computations 

Thinking in terms of strategies, there are in general more strategies a? for 
an agent i in S° than there are strategies g; for him in S. To see this 
consider the following example. 


Example 5.2. Suppose we have two agents, called 1 and 2. Suppose agent 1 
can perform three actions in q: actions a and b (which we assume are 
legal) and action c (an illegal action). Suppose agent 2 has two actions at 
his disposal in q: action d (legal) and e (illegal), and no other action to 
choose there. Suppose furthermore that the transition function 7 is such 
that 7(q,(a,d)) = t(q,(c,d)) = q. In S°, this gives rise to transitions 
T(qii(a,d)) = qi, while T(q11, (c,d)) = qo1. This enables agent 2 in S° 
to allow for strategies that depend on how lawful agent 1 behaved in the 
past in S°. For instance, agent 2 might play a kind of ‘tit for tat’ by using 
strategy o with o(qi1q11) = d, but at the same time o(q11901) = €: a legal 
action of agent 1 is replied to by a legal action of agent 2, but after an 
illegal move of agent 1, agent 2 responds by an illegal action. Notice that 
such a strategy cannot be implemented in S, since both sequences q11q01 
and q11q01 only correspond to the single sequence qq in S. 


However, as we will see, there is a way to connect computations in both 
systems directly to each other. 


Definition 5.3. We say that a computation A = qoqiq2... is compliant 
with strategy profile cag, if comp(oag, qo) = A. That is, the computation 
can be seen as the effect of applying the strategy profile cag to qo. If such a 
strategy profile exists for A we also say that A is an enforceable computation 
in the given system. 

For any state qz,...2,, E QY, let PYOjg(e1...n) be its corresponding state 
q E€ Q. Similarly, for a sequence of states s in Q°, the sequence projg+(§) 
denotes the point-wise projection of every state in the sequence to the cor- 
responding state in Q. 

Given a system S' and its associated system with good states S°, let 
A= qq... be a computation in S, and A° = sos, ... a computation in S°. 
We say that À and A° bistmulate if there are two strategy profiles, dag in S 
and oq, in 8° such that 


1. A is compliant with cag and A° is compliant with Tag 
2. for every u € N, and every i € Ag, oi(go... Afu]) = a? (so, . . . A°[ul) 
3. for every u € N, Aļu] = projg(A°[u}) 


We say in such a case also that A and A° bisimulate with strategy profiles 
Tag and Ag: Notation: (A, cag) = (A°, Cig) 
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Note that a computation need not be compliant with any strategy, and, 
moreover, if it is compliant with one, it will in general be compliant with 
several others as well. The latter is so because of two reasons: first of all, 
a computation only specifies what the transitions are within a particular 
sequence of states, and says nothing about choices on states that do not 
occur in that computation. Secondly, even within a computation, it is 
well possible that a transition from q; to qi+ı can be the effect of different 
choices by the grand coalition Ag. For two computations to be bisimilar, 
Definition 5.3 demands however that there are two strategies, one for each 
computation, in which exactly the same actions are taken, at every state in 
the computation. Moreover, item 3 guarantees that the computations also 
only visit corresponding states. 


Let an objective temporal formula p be defined as follows: 
Y :=p| 4| Yny | Oy | Oy | Yuy 


Such formulae can be interpreted on infinite paths of states A = qoqı ... in 
a straightforward way: 


S,qom--. E P iff S, qo Fp 
S, qoqi -.. = Y1 Ave if S,qoqi... = Yı and S, qoqi .-. H= Y2 
S, qoq1 - -- = OW iff Sq... HY 
S, qoqi --- H= Liv iff Vi, S,qiqi+ı --. EW 
S, qoqi --- F WU ypo iff Ji s.t. S, G41 F we and 
VO < j <i,S,qq41---F V1 


Note that, in particular, since the propositions g; are atomic propositions 
in S°, we can interpret them on an infinite path in 9°. 


Lemma 5.4. Let À = qoq1q2 ... and A° = sos182 .... Suppose furthermore 
that (A, ag) = (A°, ong). Then: 


1. for every u € N and every objective temporal formula w: 


S, Alu] EY iff S°, fu] E Y 


2. for every u € N, i € Ag, 


Afu] € L(cilqo ... Alu])) iff S°, A fu +1] E gi 


Proof. 


1. Let À and X° be as specified. Recall that, by item 3 of Definition 5.3, 
for all u € N, Alu] = projg(A°[u]) (+). We now prove by induction 
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wp that for all u € N, S,Alu] H y iff S°,A°[u] H y. For atomic 
propositions p, this follows immediately from the equivalence (*) and 
the definition of 7°. Now suppose the property is proven for w: we 
only do the O-case. Let u € N be arbitrary. 


S, Alu] = Ow iff (definition of O) 

S, Alu + 1] Ew iff (induction hypothesis) 
S°, A [u+ 1] Ew iff (definition of O) 

S°, A [u] E Ow 


2. Let Afu+1] = q, for some q € Q. Note that, by item 3 of Definition 5.3 
projg(A°[u + 1]) = q. So, A°[u + 1] iS Gey...2i_1,2:,0141,...e, for Some 
sequence £1... Li—1, Li, Zi+l;:-- Ln E {0,1}”". By definition of 7°, we 
have x; = 1 iff Afu] € &(oi (qo... Afu])). Moreover, by the truth defini- 
tion of gi, we have x; = 1 iff S°,A° — gi The result now immediately 
follows from the above two statements. 


Q.E.D. 


So, if we have two computations that bisimulate, then according to 
item 1 of Lemma 5.4, they verify the same objective temporal formulae, 
and, by item 2, a choice for an agent 7 in the original system is legal, if and 
only if in the associated system, in the state that results the proposition g; 
is true. 


Definition 5.5. Let a computation À be compatible with strategy profile 
dag. We say that coalition G behaves socially according to this profile along 
the computation A, if Vu € N, Vi € G, (A[u] € &(o;(go... Aful))). 


So, G behaves social along A, if it has a contribution to generate the 
computation A that consists only of social actions. 


Corollary 5.6. Suppose that À and A° are bisimilar computations, with 
strategy profiles oa, and Tig respectively. Let G be an arbitrary coalition. 
Then 


1. G behaves social according to cag along A iff 8°, \° = [C] good(G) 
2. If for all i € G, ci € Aj, then S°,A° E C good(G). 
Proof. 


1. Note that item 2 of Lemma 5.4 implies 


Vu E€ N,vi c G (Au € &(oi(qo.-.-Aful)) iff S°,A°[u+ 1] E gi) 
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This, in turn, implies 


(VuEN, Vie G Alu] € C(oi(go...A[ul))) iff 
Vu € N,Vi € G(S®, A [u + 1] E ai) 


The left-hand side of the above ‘iff’ states that G behaves social 
according to gag along A, and the right-hand side is equivalent to 
S°,A° = LC] good(G). 


2. This follows immediately from the above: note that if for all i € G, 
a; € Aj, then Vu € N, Vi € G Alu] € l(a; (go... A[u])). 


Q.E.D. 


The converse of item 2 of Corollary 5.6 is in general not true: if S°,A° - 
good(G), then we only know that along the computation A, all members of 
G behave well, but outside À, they may not and hence their strategy need 
not be social. 


Definition 5.7. Let \ be a computation. We say that strategies o; and o/ 
for i coincide along A, written, oi =) o, if for all u € N, o,(A[0]... Afu]) = 
o(A[O]...A[u]). For a coalition, og = oG, if for every i € G, ci =) 0%. 
Moreover, for any strategy profile cag, and 7 a strategy for i, we write 
o|t;/0;] for the strategy profile that is like cag, except that for agent i, the 


component øg; is replaced by 7;. Similarly for o[t¢/ac]. 


It is easy to see that if dag is compliant with À, and c; =) oj, then 
olo;/o;]ag is also compliant with A. 


Lemma 5.8. Suppose À is a computation, and strategy profile o is com- 
pliant with it. Suppose furthermore that G behaves social according to this 
profile along A. Then there exists a set of strategies Tg, such that tg E€ Ag 
and OG =) TG. 


Proof. For every finite prefix goq1..-dn of A, and every i € G, take 
Ti(qog1 ---Qn) = 0:(God1--- Gn). For this choice, we obviously have ci =) Ti. 
Also, note that every action 7;(qoq1---@n) is a legal choice, because i be- 
haves social according to the profile ø along A. Since by definition of a 
system S' every agent can always perform a social action, we can extend 7; 
for any other sequence § of states in such a way that the choice 7;(8) is a 
legal action. It is clear that Tg satisfies the conditions of the lemma. Q.E.D. 


We noted above that in general there are more strategies for a coalition 
in S° than there are in S. Our main result connecting a system S with 
S° now says that all enforceable computations in one of the systems have a 
computation that is bisimilar in the other. 
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Theorem 5.9. Let S and S° be as defined before. Suppose A is compliant 
with profile cag. Then: there is a computation A° in S° and a strategy 
profile ogg, such that (A, oag) ~ (A°, âg). The converse is also true: for 
every computation A° in S° that is compliant with a strategy Ang we can find 
a strategy profile oag and computation A in S such that (A, cag) = (A°, Ag) 


Proof. From S$ to S°: let A = qgq,... be an enforceable computation in S 
and let it be compliant with dag. Let o° be a strategy in S° for agent i 
satisfying: 
a; (8) = o;(projg+()) 

The strategy profile o,, generates a computation A° = sos;... for any so 
with projg(so) = qo. Hence, by definition, o,, is compliant with A°. This 
shows item 1 of Definition 5.3. By definition of this A° and o?, also item 2 
of Definition 5.3 is satisfied. We now demonstrate item 3 using induction on 
the length of the computations, u. If u = 0, A[0] = go = projg(so) = A°[0) 
(note that qo = projg(so) by our choice of so). Now suppose the following 
induction hypothesis (ih) holds: Vn < u : An] = A°[n]. Then 


Alu + 1] = by definition of computation 
T(A[u], Tag(Go -- - Afu])) = definition of 7° and ih 
T°(A°[u], Tag(So---A°[u])) = by definition of computation 
Alu + 1 
From S° to S: Let A° = sos,.... Since A° is enforceable, we know 


that A° is generated by some strategy profile o,,. Let À = projg+(A°) = 
Projg(so)projg(si)-... It is easy to see that A is generated by any strategy 
profile oa, that satisfies, for any i € Ag: 


oi(projg(so)projg(si)..-Projg(su)) = o? (S081 --- Su) 


Hence, Cag is compliant with À. Item 2 of Definition 5.3 follows directly, as 
does item 3 of that definition. Q.E.D. 


Going briefly back to Example 5.2, although the strategy 7 in S° for 
agent 2 that simulates ‘tit for tat’ has not immediately a counterpart in S, 
in a computation A°, agent 1 must have made up his mind between behav- 
ing ‘good’ and ‘bad’, and hence we either have the computation qiiqii..- 
or qiigoi1---. And those do have a counterpart qq... in S, and they are 
generated by different strategy profiles: in the first, 1 plays initially a, in 
the second, it would be c. 


5.3.2 Proving some reductions 


The following reduction holds where G acts socially and the other agents 
also act socially. 
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Proposition 5.10. Let Tw be an objective path formula and let q° be such 
that projg(q°) = 4. 


Sys, q = (G)) {Tw = Sys’, 4° = (G) ( O good(G) A ( O good(G) > TY)) 


Proof. Let w be an objective path formula. Suppose Sys, q | ((G))? Tw. This 
means that there is a social strategy ag € Ag for G, such that for any social 
strategy og € Ag for G, if A = comp((og,@),q), then Sys, \ K Tw. (x). 

Now consider an arbitrary state q° in Sys° for which projg(¢°) = q. Let, 
for every i € G, the strategy of for agent i be fixed: it is exactly like g; on ev- 
ery projected sequence, that is, let o? (5; s) be o;(projg+(s); projg(s)). Now 
consider an arbitrary strategy o& for G, and let \° = comp((ce, oe), g). If 
we can show that for any such A°, we have 


S°, ° H (O good(G JA (O good(G ) > T4) ), 


we are done. 

To show that Sys°, A° = (| good(G), recall that og is a social strategy 
for G, and then apply Corollary 5.6, item 2. To show that also Sys°, A° = 
good(G) > Tw, assume that Sys°,\° = [C good(G). Recall that A° is 
a computation A° = comp((o@,7%),q°), where o@ is fixed. To stress that 
the computation depends on the strategy a2, we will also write \°(@2,) for 
A°. Obviously, each such A°(o%) is compliant with (0%,0%). For each 
such o@, let the strategy og in Sys be obtained from o@ in the stan- 
dard way: it has to satisfy, for every agent g in the coalition G, that 
og(projg(A°[0])...projg(A°[n])) = o§(A°[O],..-A°[n]). Let Alog) = 
comp((oG,7@),q), one computation for each og. It is clear that, for each 
o%, Alog) and A°(og) are bisimilar computations, with strategy profiles 
(7a,0G) and (o%,0%), respectively. Since we assumed Sys°, A°(o%) 
good(G), by Corollary 5.6, item 1, we have that G behaves soc ac- 
cording to (og,og) along à. By Ten ma 5.8, there is a strategy TG for G 
such that Tg =) og, and tg € Ag. That is, TG is a social strategy. By (+), 
we then have Sys, A = Tw. Since À and A° are bisimulating computations, 
we also have Sys°, A° K Ty, which had to be proven. 

For the converse, let Sys°, g° H ((G))( O good(G)A( O good(G) > Ty)). 
By the semantics of ATL, this means that there is a strategy o2, for G, 
such that for all strategies o%, if A° = comp((o7%,7&),¢°), then Sys°, A° = 
(7 good(G) A ( C good(G) > Ty)). Note that o@ is fixed. From Corol- 
lary 2 we know that every strategy (ag, ag) that is bisimilar to (0%, 7) is 
such that G behaves social according to (ag, og), i.e., a E€ Ag. For any 
q E€ Q, let qq be the a corresponding state in S° with all indices being a 1. 
Now we define a strategy o;, for each i € G, as follows: 


m 
a 


Oi(God1 --- qn) = oF (G0; 91; «++ In 
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That is, g; ‘copies’ the behaviour as prescribed in of on ‘good paths’. We 
are going to show that og has the property that, if we combine it with any 
oa E Ag, every computation A = comp( (og, og), q) has the property that 
S, A | Ty. This would complete the proof that S, q H| KG} Ty. 

So, take any social strategy og for G. Take the strategy profile oag = 
(g¢,0@). Obviously, this is in Aag, since both coalitions act socially. Let 
À be the computation comp((oq, og), q). We know from Theorem 5.9 that 
there is a computation \° and a strategy profile og, such that (À, cag) ~ 
(A*, dig); Note that we use fresh symbols for this compuation and strategy 
profile, since A° and o° already have a meaning. Now we argue that \° 
can be conceived as a computation comp((7@,7@),¢°). First of all, recall 
that both G and G are acting socially in S. Hence, the computation \° 
is of the form q°qi;q2;.--- In other words, apart from possibly the first 
state, all states have indices x; that are all equal to 1! But then, on this 
computation, we can just assume that the strategy of G is the earlier o@: 
on sequences with only 1’s, we have copied the choices of o@ to øg and 
now back to \*. Formally: for all u € N and i € G: of (q°qiz 42; -- - Guz) 
= o;(49192 -- du) = oF (4° q1192; -- - Guz). Moreover, since Cag E Aag, we 


have S°,X* E [](good(G) A good(G)). But we know that on such paths, 
when they are generated by og, that Tw holds. Now we use Lemma 5.4, to 
conclude that S, A = Ty, as required. Q.E.D. 


So far we have only looked at the reduction for cases where G is acting 
socially and the other agents are also acting socially. When we alter the type 
of strategies that the agents must follow, we obtain the following reductions, 
of which the proof is similar to that of Proposition 5.10: 


Proposition 5.11. Let w be an objective path formula, and T a temporal 
operator. Let q° be such that projg(q°) = q. 


1. Sys, q = (G) TY = Sys’, 4° H (G))( C good(G) A Ty) 


2. Sys, q = (G) TY = Sys°, q° H (G))( C good(G) = Ty) 


3. Sys, q = (G) TY © Sys°, 4° H (GYTY 


There are properties we can express with the good states approach that 
we can’t express with Social ATEL. For example, the following formula: 


Sys’, der,...0n F (()) (good(Ag) U p) (5.14) 


Comparing this with (5.9) we see that both are generally saying that if 
Ag act in a social manner then y will eventually be true, however, (5.14) 
only requires Ag to follow social strategies until ọ is satisfied whereas (5.9) 
requires Ag to always act in a social manner. 
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l is forbidden in x’ 


FIGURE 4. Systems S and S’. 


We now prove that (5.14) can not be expressed in Social ATEL. Firstly, 
we introduce two systems, S and S$”, illustrated in Figure 4. Both of these 
systems consist of only one agent, which in each state is faced with the 
choice of two actions: | or r (for left or right, respectively). There is only 
one atomic proposition, p, which holds in states zı and z2 (z1 and z4 in S’). 
Both the systems are the same apart from in S” the action / is forbidden in 
x’. It does not matter if the agent chooses l or r as both of these actions 
lead to a state where the same propositions hold. This is given formally by 
the following lemma: 


Lemma 5.12. For the two systems S$ and S”, the following holds: 


a. Vye: Sy Fy S, y2 = ọ and S, z1 E ọ S, 22 E 9; 
b. YE: Sy, Fe SsS ys Ey and Si zh Ege S, E o. 


Proof. We only prove part a., as the proof of part b. follows exactly the 
same reasoning. Firstly, we argue Vp € ®: S, z1 Ey S,2z2 | p, and 
we do this using induction on y. For atomic propositions p, negation and 
conjunctions, this is straightforward. For the CGY TP case, where T is 
an arbitrary temporal operator, suppose the following induction hypothesis 
holds: S,z1 H p & S,z2 = p. To evaluate S,z1 = (G) TY we will 
consider computations that only visit z1, and, similarly, for S, z2 = KG)pTY 
gives rise to computations which only visit z2. So, using induction, the 
computations visit ‘equivalent’ states, and we are done. 
Finally, we want to prove: 


Vp EP: Sy Eyes ye 


by induction on y. The cases of p, ~y and yi A y2 are again straightforward. 

For the (G) Tọ case, notice how T(y1,1) = z1 and r(yo,l) = z1, and 
also T(y1,r) = z2 and T(y2,r) = z2. So l and r lead to the same states no 
matter whether they are performed in yı or y2. As we have already proven 
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Vyp E P: Sa E yp & S, z2 = y, it follows that S, yı = CGP TP S S, y2 H 
(GYpT P. Q.E.D. 


The following theorem illustrates the fact that (5.14) can not be ex- 
pressed in Social ATEL: 


Theorem 5.13. Given the two systems, S and S’, shown in Figure 4, 
we claim that Vp € ®, where ® is the set of all Social ATEL formulae: 
Sx H » & S’,x2’ & yg, but in the system with good states, Sjx = 
())(good(Ag) U p) while S", 2’ P£ (())(good(Ag) U p). 


Proof. We prove Vq E€ Q, Vp E€ ®: S qe y & S',d E= Y by induction on g. 
For q € {y1, Y2, 21, 22} this is clear: every q and q’ satisfy the same atomic 
propositions, and furthermore, the transitions lead to ‘similar’ states. 

Let us now look at q = x € Q. The key now is that every physical 
computation in S starting in x has a corresponding physical computation 
in S$’ starting in x’, and the same for every legal computation in S: if 
our agent chooses | in x in S, the next state is y1, which, by Lemma 5.12 
is equivalent to y2, which, by the above, is equivalent to y}. Also, every 
physical (legal) computation in S’ from a’ has a corresponding physical 
(legal) computation in S' from z. 

Finally, we show that in the system with good states, S,x = 
())(good(Ag) U p) while S’ 2’ A (())\(good(Ag) Up). Recall we only have 
one agent. In the good-state system associated with S, good(1) is true 
everywhere, and we also have S,x H (())(good(Ag) Up): along every path, 
good(1) is true until a p-state is reached. But in S$’, good(1) is false in y1’ 
and there is a computation 2’, yl’, 21’, z1’,... along which good(1)U p is 
false. Q.E.D. 


6 Conclusion 


In this paper we have extended our social laws framework further. In pre- 
vious work, the basic framework was extended to incorporate epistemic no- 
tions, but here, we have extended this by removing the assumption that all 
agents in the system will adhere to the social laws. Firstly, we extended the 
semantic structures to allow us to model physical action pre-conditions and 
legal action pre-conditions, in turn, allowing us to construct both physical 
and legal strategies for the agents. With the semantic constructs in place, 
we introduced our logical language for reasoning about such systems—Social 
ATEL—based on ATEL but extended to allow us to refer to the type of strate- 
gies being followed by the coalition of agents and the other agents in the 
system. 

Our paper is in fact only a modest first hint as to what can be done 
when allowing for social and anti-social behaviour. There are many venues 
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to further explore. For instance, our account of knowledge and belief here 
only suggests some of its use, a deep analysis is not provided here. Especially 
the notion of social belief deserves further attention. Next, connecting our 
setup to ATL and ATL* as done in Section 5 left many questions unanswered: 
we did not provide a precise fit. Finally, there are many other ways one can 
deal with anti-social behaviour. A more quantitative way of thinking for 
instance is provided in [3], where a property holds in a robust way if it 
is guaranteed even if the system does at most one non-allowed transition. 
One might also take a more qualitative view and consider some forms of 
non-social behaviour more acceptable than others. Finally, to have a norm 
as a condition on strategies rather than actions seems an interesting venue 
to explore further. 
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Abstract 


This paper is concerned with the problem of how to make inferences 
about an agent’s beliefs based on an observation of how that agent 
responded to a sequence of revision inputs over time. We collect 
and review some earlier results for the case where the observation 
is complete in the sense that (i) the logical content of all formulae 
appearing in the observation is known, and (ii) all revision inputs 
received by the agent during the observed period are recorded in the 
observation. Then we provide new results for the more general case 
where information in the observation might be distorted due to noise 
or some revision inputs are missing altogether. Our results are based 
on the assumption that the agent employs a specific, but plausible, 
belief revision framework when incorporating new information. 


1 Introduction 
1.1 Motivation 


One of the overall goals of AI research is designing autonomous intelligent 
agents that are capable of acting successfully in dynamic environments. 
These environments may be artificial or even natural. In any case, it is very 
likely that they are “inhabited” by more than one agent. So, an agent will 
in general have to interact with (some of) the others. On the one hand, 
the agent—if it does not want to be purely reactive—needs a model of its 
environment in order to make informed choices of actions that change it in 
a way that brings the agent closer to achieving its goal. On the other, it 


Giacomo Bonanno, Wiebe van der Hoek, Michael Wooldridge (eds.). Logic and the Founda- 
tions of Game and Decision Theory (LOFT 7). Texts in Logic and Games 3, Amsterdam 


154 A. Nittka, R. Booth 


also needs to model the other agents, making successful interaction more 
likely. 

Much research has been done on formalising and reasoning about the 
effects of actions on an environment. Research on an agent’s view of the 
world usually focuses on a first person perspective. How should the agent 
adapt its beliefs about the world in the light of new information? However, 
reasoning about other agents’ beliefs or background knowledge is just as 
important. This work is intended to contribute to this latter question. 

We will adopt a much narrower perspective than reasoning about other 
agents in their full complexity which includes goals, intentions, (higher or- 
der) beliefs, preferences, etc. and restrict our attention to their (proposi- 
tional) beliefs about the world. We will also forget about the dynamic 
environment and assume a static world. That is, we will work in a very 
traditional belief revision setting. But rather than answering the question 
of how an agent should rationally change its beliefs in the light of new in- 
formation, we address the question of what we can say about an agent we 
observe in a belief change process. 

In [10], the authors use observable actions to draw conclusions about 
other agents’ mental attitudes. But the beliefs of an agent manifest them- 
selves not only in its actions. They may also be observed more directly, e.g., 
in communication. So indirectly we have access to parts of other agents’ 
belief revision processes. Information they receive is their revision input, 
responses to that information are a partial description of their beliefs af- 
ter the revision. From this information we may want to reason about the 
observed agent. Consider the following scenarios. 


e We are directly communicating with another agent, i.e., we are the 
source of revision inputs for that agent. The feedback provided by the 
agent will not reflect its entire set of beliefs. To get a more complete 
picture we may want to infer what else was believed by the agent, 
what its background knowledge might be. 


e We observe a dialogue between two or more agents. Beliefs one agent 
expresses are revision inputs for the others. Due to noise, private mes- 
sages etc., we might not have access to the entire dialogue—possibly 
missing some inputs completely. So we have to deal with partial in- 
formation about the revision inputs. As we might have to deal with 
the observed agents later, forming a picture of them will be useful. 


The information at our disposal for reasoning about another agent A will 
be of the following form. We are given a (possibly incomplete) sequence of 


1 This is of course possible in the first case, as well. The communication might take place 
in several sessions and we do not know which inputs the agent received in between. 
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(partially known) revision inputs that were received by A. Further we are 
given information on what the agent believed and did not believe after hav- 
ing received each input. All this information constitutes an observation of 
the agent. First we will briefly recall results for the case where observations 
are complete with respect to the revision inputs received by A. These are 
then used for dealing with the more general case. 

The general approach to reasoning about an agent based on observations 
will be as follows. We assume A to employ a particular belief revision 
framework for incorporating revision inputs. We will then try to find a 
possible initial state of A that best explains the observation. By initial 
state we mean A’s epistemic state at the time the observation started. As 
we do not know the true initial state, we will have to select a reasonable 
one. This state explains the observation if it yields the beliefs and non- 
beliefs recorded in the observation given the revision inputs received by the 
agent. The meaning of best in this context will be explained later. The 
initial state, which can be interpreted as A’s background knowledge, will 
allow us to reason about beliefs not recorded in the observation. 

Many approaches for reasoning about action, belief revision, etc. assume 
the initial belief state being given and deal with the case of progression 
through sequences of actions/revision inputs. They say little or nothing 
about the case where the initial state is not known. In particular with 
respect to the belief revision literature this work is intended to be a step 
towards filling this gap. 


1.2 Simplifying assumptions 

We make several simplifying assumptions which will naturally limit the 
applicability of the methods developed in this work but at the same time 
allow a focused analysis of the problem we approach. 

As mentioned above, we assume a static world in the sense that the 
revision inputs and the information about the agent’s beliefs refer to the 
same world. However, it is essential for our work that the revision inputs 
were received over time. One central point is to exploit having intermediate 
steps at our disposal. The observed agent itself may only be interested in 
the final picture of the world. We in contrast want to extract information 
about the agent from the process of its arriving there. 

We restrict ourselves to propositional logic, and all components of an 
observation are already provided in propositional logic generated from a 
finite language. That is, we assume that revision inputs, beliefs and non- 
beliefs are (and are directly observed as) propositional formulae. Agents are 
assumed to be sincere, i.e., they are not deceptive about their beliefs, al- 
though the information may be partial. The observed agent will be referred 
to as A. We will disregard concepts like (preferences for) sources, compe- 
tence, context, etc. A will be assumed to employ a particular belief revision 
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framework which we describe in detail in Section 2. The only thing that 
happens during the time of observation is that A incorporates the revision 
inputs. In particular, it does not change its revision strategy or learns in 
any other way. In that sense, we consider the observations to be short term. 

We do not investigate strategies for extracting as much information about 
A as possible. The observing agent simply uses the information provided to 
reason along the way, being passive in that sense. That is, our focus is not on 
the elicitation of information about other agents; the question of optimising 
the reasoning process by putting agents in a setting where observations yield 
the most precise results is another interesting topic which we do not pursue. 

From the choice of revision framework it will become apparent that we 
equate recency with reliability of the information. We are well aware that 
this is highly debatable. We will briefly address this issue in the conclusion. 

For real world applications many of these assumptions have to be 
dropped or weakened. Many of the issues we disregarded will have to be 
taken into account. But for the moment we try to keep the number of free 
variables low in order to give more precise formal results. We hope to con- 
vince the reader that even in this very restricted setting we will be able to 
draw interesting, non-trivial conclusions. Also, we will show that even if 
these assumptions are correct, there are very strict limitations to what we 
can safely conclude about A. 


1.3 Preliminaries 


As stated above, the observed agent will be denoted by A. L will be used to 
denote a propositional language constructed from a finite set of propositional 
variables p,q,r,..., the connectives A,V,7,—,< and the symbols L for 
some contradiction and T for some tautology. a, (,6,0,A,~,¢,~, and A 
(often with subscript) will denote propositional formulae, i.e., particular 
elements of L. In Section 3, x will be used as placeholder for an unknown 
formula. + is the classical entailment relation between a set of formulae 
and a formula, where we abbreviate {a} + 6 by a | 8 for singleton sets. 
Cn(.S) denotes the set of all logical consequences of a set of formulae S, i.e., 
Cn(S) = {a | SF a}. 

The revision operation * introduced will be left associative and conse- 
quently K * y1 * p2 is intended to mean (K * ~1) * p2. o and p are used to 
denote sequences of formulae, () being the empty sequence. The function - 
denotes concatenation, so a: p and a - a represents sequence concatenation 
and appending a formula to a sequence, respectively. 

The structure of the paper will be as follows. Section 2 will introduce 
the assumed agent model as well as the formal definition of an observation. 
It further recalls the central results for the case where all revision inputs 
received by A during the time of observation are completely known, i.e., in 
particular the method for calculating the best explaining initial state and its 
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properties. The section thus summarises [5, 6, 7]. It extends these papers 
by also discussing the question of how safe conclusions we draw about A 
are. Section 3 uses these results to deal with the case where the observation 
is allowed to be more partial. In particular, some inputs may not have been 
recorded in the observation (see also [23]) and the logical content of parts 
of the observation may only be partially known. We show how this lack 
of information can be represented and dealt with. This paper is intended 
to give a broad overview over our proposed method for reasoning about an 
observed agent. Hence, we give only short proofs sketches. Full proofs are 
available in the first author’s PhD thesis [24]. 


2 Belief Revision Framework, Observation and 
Explanation 
2.1 The assumed belief revision framework 


We already mentioned that we will assume the agent to employ a particular 
belief revision framework. The first thing we will do is describe it. As we 
consider observations of A’s belief revision behaviour over time, it is obvi- 
ous that such a framework needs to support iterated revision [12, 17, 21]. 
Further, an observation may imply that a revision input was in fact not 
accepted. For example it might be explicitly recorded that after being in- 
formed that Manchester is the home of the Beatles, the agent does not be- 
lieve this statement. Consequently, the assumed revision framework should 
also account for non-prioritised revision [16, 19], i.e., revision where the 
input is not necessarily believed after revising. 

We will assume A to employ a belief revision framework [3] that is 
conceptually similar to the approaches in [4, 9, 20, 25] but is able to handle 
non-prioritised revision as well. The agent’s epistemic state [p, a] is made 
up of two components: (i) a sequence p of formulae and (ii) a single formula 
A, all formulae being elements of L. A stands for the agent’s set of core 
beliefs—the beliefs of the agent it considers “untouchable”. One main effect 
of the core belief is that revision inputs contradicting it will not be accepted 
into the belief set. p is a record of the agent’s revision history. Revision by 
a formula is carried out by simply appending it to p. The agent’s full set 
of beliefs Bel((p, A]) in the state [p, A] is then determined by a particular 
calculation on p and A which uses the function f which maps a sequence o 
of propositional formulae to a formula. This is done by starting off with the 
last element of o and then going backwards through the sequence collecting 
those formulae that can be consistently added and forgetting about the 
remaining ones. 
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Definition 2.1. 


By k=1 
FOr,- Pi) = 4 Ce NP tyes .g Bis RS LSA RINT Gp aii.) L 
f(Br-1,---, 61) otherwise 


As hinted at above, iterated revision is handled quite naturally by the 
framework. All revision steps are simply recorded and the problem of what 
A is to believe after each revision step, in particular whether the input just 
received is accepted, i.e., is believed, is deferred to the calculation of the 
beliefs in an epistemic state. In order to calculate them the agent starts 
with its core belief A and then goes backwards through p, adding a formula 
as an additional conjunct if the resulting formula is consistent. If it is not, 
then the formula is simply ignored and the next element of p is considered. 
The belief set of A then is the set of logical consequences of the formula 
thus constructed. 


Definition 2.2. The revision operator * is defined for any epistemic state 
|p, 4] and formula ¢ by setting [p, a]* yp = [p-y, a]. The belief set Bel([p, a]) 
in any epistemic state [p, A] is Bel([p, a]) = Cn(f(p- &)). 


Note, that we do not prohibit the core belief A to be inconsistent in 
which case A’s belief set is inconsistent. This is the essential difference of 
to the linear base-revision operator in [22]. From the definition, it is easy 
to see that Bel([p, 4]) is inconsistent if and only if A is inconsistent. 


Example 2.3. Consider the epistemic state [(), ~p] of an agent. The beliefs 
of the agent in this state are Cn(f(-p)) = Cn(-7p). If q is received as a 
new input, we get [(), =p] x q = [(q), =p] as the new epistemic state. The 
corresponding beliefs are Cn(f(q, =p)) = Cn(q A =p). 

A further input q — p changes the epistemic state to [(¢,q — p), =pl. 
Note, that f(q,q —> p, =p) = (q —> p) ^ =p and q cannot be consistently 
added, so now the agent believes the logical consequences of ~q A ~p. 

The revision input p changes the epistemic state to [(q,q — p,p), 7p] 
but the beliefs remain unchanged, as p contradicts the core belief. 


Given the state |p, a] of A and a sequence (%1, . . . , Yn) of revision inputs 
received in that state we can define the belief trace of the agent. This is a 
sequence of formulae characterising the beliefs of A after having received 
each of the inputs starting with the beliefs in [p, A]. 


Definition 2.4. Given a sequence (91,...,¢n) the belief trace 
(Belf, Belf,..., Bel?) of an epistemic state [p, A] is the sequence of formulae 
Belf = f(o- a) and Bel? = f(p-(¥1,...,¢i,4)), 1<i<n. 


The belief trace in the above example is (~p, q A ap, =q A ap, =q A =p). 
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2.2 Observations 


After having formalised the assumptions about any observed agent, we now 
turn to the specific information we receive about a particular agent A— 
some observation on its belief revision behaviour. An observation contains 
information about revision inputs A received, what it believed and did not 
believe upon receiving them. 


Definition 2.5. An observation o = ((~1,01, D1), -.-, (Pn, 0n, Dn)) is a 
sequence of triples (p;i, 0i, Di), where for alll <i < n: Yi, 0i, and all ô € D; 
(D; is finite) are elements of a finitely generated propositional language L. 


The intuitive interpretation of an observation is as follows. After having 
received the revision inputs y; up to y; starting in some initial epistemic 
state, A believed at least 0; but did not believe any element of D;. In 
this section, we assume that during the time of the observation A received 
exactly the revision inputs recorded in o, in particular we assume that no 
input was received between y; and ¥;+41, the observation being correct and 
complete in that sense. For the 6; and D; we assume the observation to 
be correct but possibly partial, i.e., the agent did indeed believe 0; and did 
not believe any 6 € D;, but there may be formulae w for which nothing 
is known. In this case we have both 0; ¥ w and w ¥ 6 for any 6 € Dj. 
Note that complete ignorance about what the agent believed after a certain 
revision step can be represented by 0; = T and complete ignorance about 
what was not believed by D; = Ø. 

The observation does not necessarily give away explicitly whether a re- 
vision input was actually accepted into A’s belief set or not. If 0; F y; then 
the revision input y; must have been accepted and if 6; F ay; or yi F 6 
for some 6 € D; then it must have been rejected. But if none of these 
conditions hold, it is not obvious whether an input has been accepted or 
rejected. Often, none of these two cases can be excluded. One of the aims 
of our investigation is to draw more precise conclusions with respect to this 
question. 

A given observation o = ((%1, 01, D1), .--, (Pn, On, Dn)) covers only a 
certain length of time of the agent’s revision history. When the observation 
started, A already was in some epistemic state |p, a]. We will give the formal 
conditions for an initial state to explain an observation o. The intuitive 
interpretation of o is formally captured by the system of relations in the 
second condition of the definition. 


Definition 2.6. Let o = ((y1,01, D1),.--,;(Yn,9n,Dn)). Then [p, a] ez- 
plains o (or is an explanation for o) if and only if the following two condi- 
tions hold. 
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2. For all 7 such that 1 < i < n: 
Bel([p, A] * p1 * -x yi) FO; 
and 
VO € D; : Bel([p, A] * pı * -x Gi) KO 


We say A is an o-acceptable core iff |p, A] explains o for some p. 


For us, an explanation of a given observation o is an epistemic state that 
verifies the information in o and has a consistent core belief. It is (conceptu- 
ally) easy to check whether an epistemic state [p, A] is an explanation for o. 
It suffices to confirm that the conditions in Definition 2.6 are satisfied, i.e., 
that A is consistent and that for all i we have f(p-(yi,..., pi, &)) F 6; and 
f(p- (Y1,---, i, &)) ¥ ô for all 6 € Di. A state with an inconsistent core 
belief satisfies the second condition if and only if D; = Ø for all 7, so there 
are observations that could be explained by such a state. However, we do 
not consider claiming the agent to be inconsistent worthy of being called an 
explanation. 


Example 2.7. Let o = ((p,q, Ø), (q,r, Z)) which states that A after receiv- 
ing p believes q and after then receiving q believes r. It does not inform us 
about any non-beliefs of the agent. 

[p, 4] = [(p — q),r] explains o because f(p— q,p,r) entails q and 
f(p— q,p,q,r) entails r (both are equivalent to pA qAr). [(p > q), T] 
does not explain o because f(p > q,p,¢,7) =pAqkr. [O,p^qAr], 
(p > qr), T], [(-p,4¢,7), 8], and [(q Ar), =p] are some further possible 
explanations for o. 


There is never a unique explanation for o, in fact there are infinitely 
many in case o can be explained. This is why our proposed method for rea- 
soning about A is to choose one explanation |p, a]. Using A and the belief 
trace we then draw our conclusions as follows. Revision inputs consistent 
with A will be accepted by A, those inconsistent with A are rejected. A’s 
beliefs after receiving the ith input are characterised by Bel?. In Section 2.4 
we will discuss the quality of these conclusions and present a method for 
improving them. But first we have to say how to actually choose one expla- 
nation. 


2.3 The rational explanation 


This section recalls the essential results from [5, 6, 7] for identifying and 
justifying the best of all possible explanations. A very important property 
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of the framework is that A’s beliefs after several revision steps starting in 
an initial state can equivalently be expressed as the beliefs after a single 
revision on the same initial state. Intuitively, the agent merges its core 
belief and all revision inputs received using f into a single formula and then 
conditions its epistemic state using it. 


Proposition 2.8. Bel((p, a] * yi *--- * yi) =Bel([p, a] * f(y1,---, Pi, A)). 
Proof (Sketch). Note that by Definition 2.4 it suffices to show that 


fle: (P1,-++, i &)) = flo (F(e1,---, Pi, A), A)). 


One property of f that follows from its recursive definition is f(o-o’) = 
f(o- f(o’)). If we can show that f(p1,..., pi, A) = f(f(gi,---, Pi, A), A) 
we are done as then in both cases equivalent formulae have been collected 
before processing p. We can restrict our attention to consistent A, in which 
case f(~1,..-, i, A) is consistent and entails a. Hence 


FCCP- Pi A), A) = f(G1,---, Pi A) AN A= f(yi,---, Gi, A). 


Q.E.D. 


How does that help to reason about the observed agent A? Recall that 
an observation o = ((~1, 61, D1), --- , (Pn, On, Dn)) expresses the information 
that “revision by ¢ in the initial state leads to a new state (where 0, but 
no element of Dı is believed) in which revision by 2 leads to...” That 
is, the observation contains bits of information concerning beliefs and non- 
beliefs in different (if related) epistemic states. This proposition now allows 
us to translate the observation into information about a single state—the 
initial epistemic state we are after. Note however, that A needs to be 
known for applying the proposition as otherwise f(y1,..., pi, &) cannot be 
calculated. So, given a core belief A, o yields that A would believe 6; (and 
would not believe any ô € D;) in case it revised its initial epistemic state 
by f(p1,.--, Pi, A). This is nothing but conditional beliefs held and not 
held by A in its initial state [p, a]. That is, o is a partial description of A’s 
conditional beliefs in |p, a]. The proposition further entails that if we had a 
full description of its conditional beliefs we could calculate the beliefs after 
any sequence of revision inputs. 

It turns out that the assumed belief revision framework allows us to 
apply existing work ([18] and in particular [8]) on completing partial infor- 
mation about conditional beliefs? and to construct a suitable p such that 

2 [8] presents a rational closure construction that takes into account both positive and 


negative information as is necessary in our case. It extends the case of positive-only 
information studied in [18]. These papers also inspired the name rational explanation. 
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[p, 4] is indeed an explanation for o in case A is o-acceptable. pr(o, A) 
denotes the sequence thus constructed. The construction even reveals if a 
given core belief is o-acceptable. 

We further showed that the set of o-acceptable cores is closed under dis- 
junction. If A, and A» are o-acceptable, then so is A, V A2.” This entails 
that—if o can be explained at all—there is a unique logically weakest o- 
acceptable core belief, which we denote by Ay(o). Consequently A F Ay(o) 
for any o-acceptable A. The rationale behind choosing Ay(o) for an ex- 
planation is that any input we predict to be rejected by A will indeed be 
rejected. Furthermore, it can be shown that adding beliefs or non-beliefs 
to o by strengthening some 6; or enlarging some D; as well as appending 
observations to the front or the back of o to get an observation o’ cannot 
falsify this conclusion as Ay(o’) F Ay(o). For any other core belief explain- 
ing o, a revision input predicted to be rejected by the agent might in fact 
be accepted. In this sense, we consider Ay(o) to be optimal. 

The choice of pr(o, Ay(0)), which we call the rational prefix, as the se- 
quence in the agent’s initial epistemic state is justified by showing that it 
yields an optimal belief traces. Let p = pr(o,av(o)) and ø be the se- 
quence of any other explanation |ø, Ay(o)| for o, (Belf, Bel?,..., Bel?) and 
(Bel§, Belf,..., Bel?) be the corresponding belief traces. Then the following 
holds: If Bel? = Bel? for all j < i then Bel? H Bel?.* This tells us that the 
formulae we predict the agent to believe initially will indeed be believed 
(although some further formulae might be believed as well)—provided the 
agent’s core belief really is Ay(o). And if our predicted belief trace exactly 
captures the agent’s beliefs up to the ith input then again all beliefs pre- 
dicted after the next input will indeed be believed. The assumption that 
the two explanations use the same core belief causes this criterion, which 
we will refer to as the optimality criterion for the rational prefix, to be a 
rather weak one as we will see shortly. 

In [7], we defined [pr(o, Av(0)), Av(o)] to be the rational explanation 
of an observation o—if there is an explanation at all. That paper and [5] 
contain more results about the rational explanation but these are the most 
important ones which justify the claim that the rational explanation is the 
best explanation for a given observation o. An algorithm which calculates 
the rational explanation is given below and described in more detail in [6]. 
The problem with calculating [pr(o, Ay (0)), Av(0)] is that Ay(o) has to be 
known, which it is not in the beginning. So the idea is to iteratively refine 
the core belief starting with the weakest possible of all T. 


3 The proof is constructive and not deep but lengthy. 

4 This result, as almost all the others in this section, is proved in [5]. Note also that 
Bel? + Bel? need not hold. Consider [(-p), T] and [(p A q, =p), T]. The belief traces 
when assuming a single input p are (~p, p) and (~p, p ^ q). Although the beliefs are 
equivalent initially, they need not be after a revision step. 
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Algorithm 1: Calculation of the rational explanation. 


Input: observation o = ((~1, 01, D1),.--, (Pn, On; Dn)) 
Output: the rational explanation for o 


AT 

repeat 
p = pr(o, A) /* now p= (Qam, ...,@0) */ 
A < A NAAm 

until am = T 

return |p, A] if A Æ L, “no explanation” otherwise 


Having calculated the rational explanation [p,a] of an observation 
o = ((y1,41, Di),---, (Yn, On; Dn)), we can make predictions concerning 
which inputs A accepts and rejects based on A and our conclusions about 
its beliefs after having received each input are summarised by the corre- 
sponding belief trace (Belf, Bel?,..., Bel?). 


2.4 Safe conclusions and hypothetical reasoning 


In the remainder of this section, we will illustrate some limitations of the 
rational explanation. In particular, we will show that predictions based on 
it will almost never be safe ones. However, this is inherent in the problem 
and not due to our solution to it. 

As with many optimisation problems the quality of the solution and the 
conclusions we can draw from it depend heavily on the quality of the data 
and validity of the assumptions made. In our case, we clearly stated the 
assumptions made about the given observation as well as the agent’s being 
ruled by the assumed framework. The optimality result for the best ex- 
plaining core belief Ay(o), i.e., that Ay(o) is entailed by any o-acceptable 
core, depends on those. The optimality of the rational prefix pr(o, A) and 
therefore the conclusions about A’s further beliefs also depend on its actu- 
ally employing the assumed core belief. That is, if we cannot be sure of the 
agent’s actual core belief then most of what we can say about the agent’s 
belief trace based on the rational explanation is merely justified guesses but 
not safe bets. 


Example 2.9. (i) Let o = ((T,p, Ø), (~p, T, Ø), (r = 7p,r V p,@)). The 
rational explanation for o is [(p), T] and the corresponding belief trace is 
(p,p,7p,r A ap). That is, we conclude that A accepted the input ~p and 
believes r A ~p after then receiving r << ~p. 

Now assume the agent’s real initial belief state was [(),p]|—note that 
the core belief does not correspond to the one calculated by the rational 
explanation—and thus the belief trace in truth is (p,p,p,7r A p). That is, 
it did not accept the input ~p and believed ~r A^ p after receiving r < 7p. 
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So except for the beliefs before the observation started and after receiving 
the tautology (where we are informed that the agent believes p and hence 
must have believed it initially) most of the conclusions about beliefs held 
by A we draw from the belief trace are wrong! 

(ii) Let o = ((p,p, 2), (4,4,2), (r = p,T,@)). The rational expla- 
nation for o is [(), T] and the belief trace implied by that explanation is 
(T,p,p^q,p^AqAr). Assuming that [(),q —> 7p] was A’s true initial state, 
the belief trace in truth is (q > ~p, p A 7q,q A 7p,q A 7p A nr). Again, for 
large parts the conclusions we draw about the agent’s beliefs based on the 
rational explanation are wrong. For example, we conclude that agent con- 
tinues to believe p once it has been received. This is clearly not the case. 


This strong dependence on the core belief can be easily explained. There 
are two main effects due to the core belief. First, it causes revision inputs to 
be rejected immediately. This is why the conclusions based on the rational 
explanation are off the mark in case (i) in the above example. Secondly, 
the core also accounts for interactions between revision inputs. An earlier 
input is eliminated from the belief set in the light of the core and some 
later inputs. This effect is illustrated in case (ii). For one choice of the core 
belief, after having received the input yi+;, the agent may still believe the 
input y; received earlier, while for another core it may believe 7y;. 

Even if we got the core belief right and hence the agent really employs 
Av(o), conclusions based on the rational explanation of o should not be 
used without care. The optimality result for the rational prefix does not 
exclude mistakes. Correct conclusions about beliefs are guaranteed only up 
to the point in the belief trace where the beliefs we calculate and the agent’s 
actual ones first fail to be equivalent. This can easily be the case already 
for the initial beliefs. 

Consider o = ((p,q, Z), (r, T, Ø)) for which the rational explanation is 
[(p — q), T], the corresponding belief trace being (p > q,p\q,pAqA7). 
So we would conclude the agent to keep believing in q. If the agent’s real 
initial epistemic state was [(-q, =r A^ q), T] then the real belief trace would 
be (ar Ag,pAn7r ^q, r Anq). Although the correct core was calculated, we 
would still be wrong about q whose negation is in fact believed after having 
received the input r. 

As stated above, using the rational explanation [p, Ay(o)] we conclude 
that A believed Bel? = f(p-(y1,..., Pi, Av(0))) after having received the 
first 2 revision inputs recorded in o. How safe is this conclusion? The above 
example showed that it is not very safe. So what can we do to further 
improve the results? 

Here, we will consider only one very strong notion. We call the con- 
clusion that A believes w after receiving the ith revision input recorded 
in o safe if and only if for all explanations for o we have Bel; F Y, where 
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Bel; is the element of the belief trace corresponding to that input. In other 
words, every possible explanation predicts that belief (so in particular the 
one corresponding to the agent’s real initial state). Analogously, we call the 
conclusion that the agent did not believe y% at a certain point safe when- 
ever no explanation predicts that formula to be believed. Note that a safe 
conclusion about an agent’s belief does not mean that this belief is correct. 
The agent may have received and accepted unreliable information, but it 
means that given the observation, the agent must have held this belief. 

We will now describe a way to calculate whether a conclusion of that 
form is safe, a method we call hypothetical reasoning. By this we mean mod- 
ifying the given observation according to some conjecture and rerunning the 
rational explanation construction on the observation thus obtained. Note 
that any explanation for 


o' = ((~1, 61, D1),. -< (Yi, 0i Av, Di), . reg (Pn, On, Da)? or 
d = ((p1, 01, D1), ---, (Pi, bi, Di U {Y}, ---, (Pn, On, Dn)) 


will also explain o = ((y1,01,D1),..., (Pi, 0i, Di),---, (Pn, On, Dn)). This 
follows directly from Definition 2.6. If 0; \w belongs to the beliefs after the 
ith revision step then so does 6; and if none of the elements of D; U {4} is 
believed at that point, the same holds for any subset. 

So in order to check whether the conclusion of A believing w after re- 
ceiving the ith revision input is a safe one, we simply add w to D; and 
test whether the observation thus obtained has an explanation.® If so, then 
the conclusion is not safe as there is an explanation where w is in fact not 
believed. However, if no such explanation exists then Y% must indeed be be- 
lieved by A. The non-belief of a formula 7 can be verified by replacing the 
corresponding 6; by 6; A w. If the observation thus obtained has an expla- 
nation then the agent may have believed 7 and consequently the conclusion 
is not safe. 

With a small modification this method works also for hypothetical rea- 
soning about the agent’s initial beliefs, i.e., before receiving the first input. 
It does not work directly, as the observation does not contain an entry for 
the initial state. By appending ((Yo, 40, Do)) = ((T, T, @)) to the front of 
the observation o we create this entry. The point is that receiving a tau- 
tology as input leaves the beliefs unchanged. We can now add a formula w 
to Do or fo as described above to verify conclusions about the initial state. 
Further, there is no restriction which formulae w~ can be used for hypothet- 
ical reasoning. It is even possible to add several 7; simultaneously to o to 
get a modified observation o’. 


5 The rational explanation algorithm always finds an explanation if there is one, and 
returns “no explanation” if there is none. 
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Hypothetical reasoning can also be used in order to improve the con- 
clusions about A’s core belief A. We already know that a+ Ay(o), i.e., 
all inputs we predict to be rejected by A will indeed be rejected. This is 
because any o-acceptable core entails Ay(o). But what about the other in- 
puts, must they really have been accepted? Can we be sure that y; really 
was accepted if it is consistent with Ay(o)? Rejecting y; is equivalent to 
not believing the input after having received it. So, we simply add y; to 
Dj, i.e., replace (y;,6;,D;) in o by (Yi, 0i, Di U {yi }) to get o’. If there is 
an o’-acceptable core, then A may in fact have rejected y;. However, if o’ 
does not have an explanation then we know that A must have accepted that 
input. 

It might be nice to be able to check whether conclusions about A’s beliefs 
are safe, but can we ever be sure to have the correct core belief in order to 
apply the optimality results we gave for the rational prefix? The answer to 
this question is almost exclusively negative. Usually, there is more than one 
o-acceptable core. In a different context Sébastien Konieczny® suggested the 
additional assumption that the last belief 0„ recorded in the observation 
o is in fact complete. This assumption gives us an upper bound on the 
actual core belief A as then 0n F A F Ay(o) must hold and we can use 
the hypothetical reasoning methodology in order to get an improved core 
belief. As we know the exact belief 0, at the end of the observation, we 
can iteratively add to D, those formulae 7 which the rational explanation 
predicts to be believed but which are not entailed by 6,,. This method will 
yield an improved lower bound for the core belief of the agent, but it cannot 
guarantee uniqueness of the core. 

Even if we assumed that every 0; completely characterises the beliefs of 
the agent after receiving y;, we would not be guaranteed to get the real core 
belief. Consider o = ((p, p, Ø), (q, p ^q, 2), (r,p ^q Ar, Ø)) to illustrate this. 
The rational explanation for o is [(), T]. However, p is also an o-acceptable 
core, |(), p] being one possible explanation. That is, the conclusion that an 
input —p will be accepted by the agent is not safe. This illustrates that even 
using much more severe assumptions about a given observation, identifying 
the agent’s real core belief is impossible. 


3 Extension to Unknown Subformulae 


Up to this point we considered observations that were complete with respect 
to the revision inputs received. We knew exactly which inputs were received 
during the time of observation. The scenarios in the introduction suggested 
that it is well possible that some of the inputs might have been missed. Fur- 
ther, the observer may not understand the complete logical content of all the 
revision inputs, A’s beliefs and non-beliefs. Consider the following example 


6 Personal communication. 
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where the agent is observed to receive exactly two inputs p and q. After 
hearing p, the agent believed something we cannot understand, but after 
then hearing q, it did not believe that anymore. In the original framework 
this cannot be formalised as there is no means to represent the unknown 
belief. However, we should be able to conclude that A believed ~q after 
having received p. This is because the assumed belief revision framework 
satisfies (most of) the AGM postulates [1]. In particular, if the input is 
consistent with the current beliefs they have to survive the revision process 
(cf. the “Vacuity” postulate from AGM) which is clearly not the case in 
the example. The current section investigates how the previous results can 
still be used to reason about A if the observations are allowed to be less 
complete in this sense. 

We want to emphasise that there is a big difference between knowing 
there was a revision input while being ignorant about its logical content 
and not even knowing whether there were one or more revision inputs. We 
will first deal with the former case which will provide results to deal with 
the latter one in Section 3.3. 


3.1 Modelling unknown logical content 


We will model partial information by allowing formulae appearing in the 
observation to contain unknown subformulae which are represented by n 
placeholders xj. A(X1,---, Xn )[(Xi/Qi)i] denotes the result of replacing in A 
every occurrence of x; by Qi. 


Definition 3.1. Let L be a propositional language and yj,..., Xn be place- 
holders not belonging to L. 

A “formula” A(y1,---;Xn) possibly containing x1,...,X%n is called a 
parametrised formula based on L iff A(x1,---,Xn) [Oxi /)i] E€ L whenever 
$ € L. o = ((p1ı,01, Dı), ---, (Yı, 01, Di)) is a parametrised observation 
based on L iff all y;, 0i, 6 € D; are parametrised formulae based on L. 
We denote by L(o) the smallest language L a parametrised observation o is 
based on. 


To put it differently, a parametrised formula based on L is a formula from 
L in which some subformulae have been replaced by placeholders y;. This 
allows hiding parts of the logical content of a formula. So in order to model 
(even more) partial knowledge, we will consider parametrised observations. 
The example from the introductory paragraph can now be represented by 
o = ((p, x, 2), (q4, T, {x})). We will often write \ rather than A(X1,---, Xn) 
to denote a parametrised formula in order to ease reading. 

Unknown subformulae x; are allowed to appear in all components of 
an observation—revision inputs, beliefs and non-beliefs. The same y; can 
appear several times. In fact, this is when it contributes to the reasoning 
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process. It is not unreasonable to assume that this can happen. For ex- 
ample, the meaning of an utterance in a dialogue might not be understood 
as part of the language may not be known to the observing agent, but the 
utterance might be recognised when it appears again later. Analogous to 
a learner of a foreign language, the observer may be familiar with (parts 
of) the structure of the language while being ignorant about the meaning 
of certain “phrases”. In case we are completely ignorant about the logical 
content the entire parametrised formula will simply be a placeholder. 

Let o be a parametrised observation. o[y1/¢1,.--,;Xn/@n] and equiva- 
lently o[(vi/¢:);] denote the observation obtained by replacing in o every 
occurrence of the placeholder x; by a formula ¢;. 

We still assume correctness of the information contained in the parame- 
trised observation o, i.e., we assume the existence of instantiations ¢; of all 
unknown subformulae y; such that the observation o[(y;/@;);] is a correct 
observation in the sense of Section 2—in particular, there must be an entry 
for every revision input received. The agent indeed received exactly the 
inputs recorded and beliefs and non-beliefs are correct if partial. Note that 
this implies that we are not yet able to deal with missing inputs. These will 
be considered in Section 3.3. One important technical restriction is that 
the instantiations of unknown subformulae x; must not contain unknown 
subformulae x; themselves, i.e., the instantiations must be elements of the 
underlying language—however, not necessarily elements of L(o). That is, 
the true meaning of x; is not assumed to be expressible in the language of 
the known part of o. Abusing notation we will write that o has an expla- 
nation, meaning that there exist instantiations ¢),...,@, for the unknown 
subformulae such that o[(v;i/¢;);] has an explanation; similarly that A is 
o-acceptable if A is o[(yi/¢;);]-acceptable. 


3.2 Finding an acceptable core belief 


In this section, we will present results on what can be said about A’s core 
belief given a parametrised observation o. If an explanation exists at all, 
once more there will be a unique weakest o-acceptable core A. This may 
be surprising as there are many different possible instantiations for the un- 
known subformulae. But this will also allow us to choose them such that any 
o-acceptable core entails A. If we knew the instantiations of the unknown 
subformulae we could simply use the rational explanation algorithm, as in 
that case a parametrised observation could be transformed into a regular 
one. As we do not know them, we have to guess. The trick is to extend the 
language and treat every y; as a new propositional variable 2;. 


Proposition 3.2. If [p, a] explains o[(x;/@;);] and z1,...,£n are prop- 
ositional variables not appearing in o, A, p or any ¢; then of(xi/zi)i] is 
explained by [p, A A Aj <i<n(xi  ¢i)]- 
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Proof (Sketch). Al(xi/di)i] F L if Nicicn(ti > Qi) AALOG/zi)i] F -L for 
any parametrised formula A not containing x; is the key to this result. As 
the x; are not contained in [p, A] or o[(xi/¢:):], requiring A(x: > ¢;) en- 
sures that the different instantiations have the same logical consequences— 
modulo entailment of irrelevant formulae containing the x;. The (relevant) 
beliefs are the same for both explanations. Q.E.D. 


The proposition formalises that given there is some instantiation for 
the unknown subformulae in o such that the resulting observation has an 
explanation, we can also replace them by new variables and still know that 
there is an explanation. However, this tells us that we can apply the rational 
explanation algorithm to oļ|(xi/x£i):] and be guaranteed to be returned an 
explanation if there is one. If this fails, i.e., we are returned an inconsistent 
core belief, then no explanation can exist using any instantiation of the 
unknown subformulae in o. The core belief A A A, <;<,,(@i > ĝi) ensures 
that the new variables x; behave exactly as the “correct” instantiations ¢; 
for the unknown subformulae in o. 

In general, Avy (o[(xi/2:)i])—the core belief returned by the rational ex- 
planation algorithm—will not be that particular formula. Note that this 
would be impossible as there can be several suitable instantiations such 
that o[(yi/¢;);] has an explanation. The core belief calculated will in gen- 
eral be weaker but may still contain (some of) the additional variables 2;. 
We will now go on to show that it is possible to eliminate these variables 
from the core belief by choosing different instantiations for the unknown 
subformulae. 

The idea is to split the core A calculated by the rational explanation 
construction into two parts, one a’ that talks only about L(o) and not 
at all about the additional variables and one w part that talks also about 
those. Formally, we choose a’ and w such that A = a’ AW and Cn(a’) = 
Cn(4) N L(o), which is possible as we are in a finite setting.” Instead of x; 
we then use x; A wv to instantiate the placeholders. This shifting of parts of 
the core to the new variables is possible because the part of the core belief 
that talks about the z; becomes relevant in the calculation of the beliefs of 
an agent only when those variables themselves appear. 


Proposition 3.3. If [p, a] explains o[(x;/x;);] then there exist a’ and w 
such that a’ contains no x; and [p-w, a’] explains of|(xi/zi A Yil. 


Proof (Sketch). Let Y = A and a’ such that Cn(a’) = Cn(4) N L(o). It 
can now be shown that f(yi[(xi/a:)i],---, 9 [(Xe/22)i], A) is equivalent to 
F(a, Gi [(xi/ti ^A), 5 [(xi/ts A &):], 4’). Again, the proof of that is 


T We can trivially choose p = A and a’ to represent all logical consequences of A in 
L(o). 
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not deep but lengthy. The intuition is that a’ makes sure that with respect 
to L(o) all formulae are treated correctly and using «;/\ rather than just zi, 
the effect of A with respect to new variables is maintained. Consequently, 
before processing p when calculating the beliefs, in both cases equivalent 
formulae have been constructed and the beliefs will thus be equivalent. 

Q.E.D. 


To summarise what we know so far. Given a parametrised observation 
o has an explanation, we can construct one for o[(yi/x;);|.. However, the 
corresponding core belief Avy(of|(xi/xi):]) may still contain variables that 
are not contained in L(o) and thus we cannot claim that the agent had 
contact with them. The last proposition now showed that we can construct 
instantiations for the unknown subformulae such that the explaining core 
belief a’ is in L(o). We can even go one step further and show that any 
ol(vi/¢i)i]-acceptable core A” must entail the core a’ constructed as de- 
scribed above. 


Proposition 3.4. Let [p”, a”] be an explanation for o[(x;/¢;);] and [p, 4] 
be the rational explanation for o[(yi/2i);], where x; are additional propo- 
sitional variables not appearing in any ¢;, A” or the language L = L(o). 
Further let a’ such that Cn(a’) = Cn(a)O L. Then A” F a’. 


Proof. By Proposition 3.2 a” A N(z;i > gi) is o[(yi/xi);|-acceptable and 
hence entails A (any o-acceptable core entails Ay(o)). Obviously A F a’, 
so A” A A(z; > ¢;) F 4’. Now assume A” does not entail a’ which implies 
there is a model for A” \ 7a’. Neither A” nor A’ contain any x; so we can 
extend that model to one for a” A A(z; © ¢;) Ana’ by evaluating x; just 
as ¢;—contradicting a” A A(z; e Qi) F a’. Q.E.D. 


There is an important consequence of that result. As in the original case 
there is a unique weakest o-acceptable core for a parametrised observation o. 
This follows directly from the last two propositions. A’, being constructed as 
described above, is o-acceptable and is entailed by any o-acceptable core, so 
in particular by the agent’s real core belief. Hence, all formulae inconsistent 
with a’ will be rejected by A. That is, a’ yields a safe conclusion with 
respect to which formulae must be rejected by A—no matter what the 
instantiations of the unknown subformulae really are. 


Example 3.5. Consider o = ((x, X, Z), (p,q A =x, 2)). This parametrised 
observation expresses that the observed agent accepted an input whose 
meaning is unknown to us. After then receiving p, it believed q and the 
negation of the unknown input. The observation constructed according to 
Proposition 3.2, where x is replaced by a new variable gz, is 
olx/z] = ((#, 2,2), (p,q A =x, @)). The rational explanation for o[,/x] is 
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[((p A 7a > q),p > 7a] and (p > (~x Aq), £ A 7p,p Aq A 72) is the 
corresponding belief trace. 

This indicates that after receiving the unknown input the agent believes 
ap. In order to test whether this is necessarily the case, we investigate the 
parametrised observation o! = ((x, x,{—7p}), (p,q A ay, @)). According to 
the hypothetical reasoning methodology, =p was added to the non-beliefs. 
Applying the rational explanation algorithm yields that o’[,/z] has no ex- 
planation. Proposition 3.2 now tells us that there cannot be an explanation 
for o'—no matter how y is instantiated. That is, if the parametrised obser- 
vation correctly captures the information about the agent, it must believe 
ap after receiving the first input. 

o is based on the language L constructed from the variables p and q 
and Cn(p > ~x) N L = Cn(T). To illustrate Propositions 3.3 and 3.4 
note that o[y/x A (p > 72)] = ((a@ A =p, £ A =p, Z), (p,q A (az V p),@)) is 
explained by [(pA 7a — q, p — 72), T], the corresponding belief trace being 
(p > (AaAq),c£An7p,pA\qA72). T is trivially entailed by any o-acceptable 
core. 


In order to find an acceptable core for a parametrised observation o, 
we extended the language L(o) with new variables. In [23], we gave an 
example—which we will not repeat here—illustrating that there are param- 
etrised observations that have an explanation when language extension is 
allowed but which cannot be explained restricting the language to L(o). In 
other words, the proposed algorithm of replacing each x; by a new vari- 
able x;, running the rational explanation construction and then eliminating 
the x; from the core belief (the result being a’) may yield an explanation, 
although none exists when restricting the instantiations of the x; to L(o). 
Although we know that each acceptable core will entail a’, we cannot gener- 
ally say that restricting the instantiations to L(o) will allow the same core, 
a strictly stronger one or none at all to explain o. 

Note that Proposition 3.2 makes no assumption about the language of 
the instantiations ¢; of the unknown subformulae. They may or may not 
belong to L(o). They may contain arbitrarily (but finitely) many propo- 
sitional variables not belonging to L(o). However, that proposition has 
an interesting implication. It says if o[(yi/¢;);] has an explanation then 
so does o|(xi/xi)i], but o[(yi/xi);] contains only variables from L(o) and 
n additional variables x;, one for each placeholder. As that observation 
has an explanation, the rational explanation construction will return one. 
However, that construction uses only formulae present in the observation. 
Consequently, it does not invent new variables. So, no matter how many 
variables not appearing in L(o) were contained in the ¢;, n additional vari- 
ables suffice for finding an explanation for the parametrised observation o. 
This yields an upper bound on additional variables needed. 
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In Section 2.4 we showed that assuming the wrong core belief greatly 
affects the quality of the conclusions about A’s other beliefs. And even if 
the core is correct, the belief trace implied by the rational explanation does 
not necessarily yield only safe conclusions with respect to the beliefs of the 
agent during the observation. 

These problems are obviously inherited by the current extension to par- 
tial information about the logical content of the formulae in an observation. 
They cannot be expected to become less when not even knowing what in- 
puts the agent really received or when information about the beliefs and 
non-beliefs becomes even more vague. Much depends not only on the core 
belief but also on the instantiation of the unknown subformulae. So rather 
than just having to calculate a best initial epistemic state, we now would 
also have to find an optimal instantiation of the unknown subformulae. 
However, the limitations illustrated in Section 2.4 prevent us from even at- 
tempting to look for them. Instead, we propose to investigate the belief 
trace implied by the rational explanation of an observation o[(x;/x:);] and 
reason hypothetically about beliefs and non-beliefs from L(o) in that belief 
trace. 


3.3 Intermediate inputs 


Up to now, we assumed the (parametrised) observation o to contain an entry 
(y, 8, D) for every revision input received by A, even if some of the formulae 
are only partially known. This corresponds to the assumption of having 
an eye on the agent at all times during the observation. In this section, 
we want to drop this assumption. That is, we will allow for intermediate 
inputs between those recorded in o. In real applications this will be the 
norm rather than an exceptional case. A or the observing agent may leave 
the scene for a time, and if the observing agent is the source of information 
then o might have been gathered over several sessions between which A may 
have received further input. 

Using our notation for observations, an intermediate input is one we 
have no information about, i.e., we do not know what the revision input 
is or what is believed or not believed after receiving it. Hence, we can 
represent it by ((v, T, )); x again represents an unknown formula. Note 
that this is different from ((x,,@)) as here the input would be required to 
be accepted by A. In other words, the agent’s core belief would have to be 
consistent with the instantiation of x. 


Example 3.6. Consider the following observation without intermediate in- 
puts: o = ((p,q, Ø), (p, 7g, @)). Assume A was o-acceptable and thus con- 
sistent. Then either it is consistent or inconsistent with p. In both cases, 
the belief set does not change upon receiving the second input p. Either 
the first p was accepted and hence already believed or p was rejected (both 
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times) in which case the belief set never changes. So ~q must have been 
believed already after the first p was received. But it is not possible to 
believe q A ~q consistently (the belief set is inconsistent if and only if the 
core belief is inconsistent). Consequently, there is no o-acceptable core. 

Assuming a single intermediate input ((x, T, @)), there is only one rea- 
sonable position yielding o’ = ((p, q, Ø), (xv, T, Z), (p, ~q, Z)). Instantiating 
the unknown formula x with p > ~q, [(p > q), T] is an explanation. Be- 
fore receiving the first input the agent believes p — q, after receiving the 
first p it believes p ^q and after receiving the (assumed) intermediate input 
as well as after receiving the last input p it believes p A ~q. Hence T is 
o’-acceptable. That is, while o does not have an explanation, assuming an 
intermediate input allows the observation to be explained. 


In the general case we do not know how many intermediate inputs were 
received at which points in a given (parametrised) observation o. In [23] 
we showed that number and positions of the intermediate inputs have an 
impact on the possible explanations of o. If number and positions are 
fixed then we deal with a parametrised observation (containing an entry 
for every revision input received) and can hence use the results of Sec- 
tion 3.2 in order to calculate the weakest acceptable core belief. To rep- 
resent the intermediate inputs we simply have to introduce further un- 
known subformulae not contained in o. Assume we have the partial ob- 
servation o = ((p,q A x1, Z), (r, 79, 2), (p,q, {X1})) and the information 
that exactly two intermediate inputs have been received immediately af- 
ter r. In order to reason about A, we consider the partial observation 
od = ((p, q\X1; Ø), (r, “q, Ø), (x2, T, Ø), (x3, T, Ø), (p, q, {x1})) which now 
contains an entry for every input received. At this point we want to em- 
phasise once more that intermediate inputs and partial information about 
inputs are related but distinct cases. 

In the following we want to indicate what can be said about the agent’s 
core belief depending on how much information we have concerning possible 
intermediate inputs. Naturally, the more specific our knowledge concerning 
number and positions, the more informative the conclusions can be. We will 
start with the case where we have no information at all, which means that 
any number of intermediate inputs may have been received any time. Then 
we will turn to the cases where the positions or the number are restricted. 
Any number of intermediate inputs at any time. Consider an ob- 
servation o = ((41,01, D1), ..., (Yn; On, Dn)). Assume [p, A] explains the 
observation o’ which is obtained from o by putting some arbitrary number 
of intermediate inputs at any position in o. It can be proved that then there 
are a sequence g and n — 1 intermediate inputs such that |ø, a] explains o” 
obtained from o by putting exactly one intermediate input between any two 
inputs y; and yi41 in o. Note that both explanations use the same core 
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belief. Intuitively, the intermediate input from o” before the input Yi+1 is 
the conjunction of all relevant intermediate inputs from o’ before that input. 

Proposition 3.2 tells us that we can also use new variables x; instead of 
those intermediate inputs® and the observation o" thus obtained is guar- 
anteed to have an explanation. However, o" does not contain any un- 
known subformulae, so we can apply the rational explanation construc- 
tion which will return some epistemic state with core belief a’. We can 
now construct the weakest possible core belief by taking a” such that 
Cn(a”) = Cn(a’)OL(o). Any o’-acceptable core belief—o’ being constructed 
as described above—will entail a”. That is from a” we can safely conclude 
which formulae are rejected by A, no matter how many intermediate inputs 
it received at any point during the observation. 

What happens if we have further information about the positions or 
the number of intermediate inputs? The following proposition implies that 
we should always assume the maximal number of intermediate inputs. It 
says that an additional intermediate input, which we instantiate with a new 
variable for calculating the weakest possible core belief, can only make the 
core logically weaker. Conversely, not assuming the maximal number of 
intermediate inputs may lead to the conclusion that A rejects a formula 
which it actually does not reject simply because an additional intermediate 
input allows A’s core belief to be logically weaker. 


Proposition 3.7. If Cn(a) = Cn(ay(o1 - ((x, T, S)) + 02)) N L(o1 : 02) and 
x ¢ L(01- 02) then Ay(01-+02)F A. 


Proof (Sketch). By showing Ay(01 - 02) = Av(o1- ((T, T,@)) - 02), which 
holds because a tautologous input has no impact, we introduce the extra 
input which allows us to compare the cores. By Proposition 3.2, 
Ay(01-((T,T,@)) +02) Ax is 01- ((x, T, B)) - 0g-acceptable and hence en- 
tails Ay(o1 : ((x, T, Ø)) - 02). We can now show that any formula from 
L(0, + 02) entailed by that core as already entailed by Av (01 - 02). Q.E.D. 


Fixed positions of intermediate inputs. Now assume we know the po- 
sitions where intermediate inputs may have occurred. This is imaginable, 
for example, in scenarios where the observing agent gathers o in several ses- 
sions, but does not know if A receives further inputs between those sessions. 
How many intermediate inputs should be assumed at each of those points? 
We cannot allow an arbitrary number as this is computationally infeasible, 
so it would be helpful to have an upper bound which we could then use. We 
claim that it suffices to assume j intermediate inputs at a particular posi- 
tion in o, where j is the number of revision inputs recorded in o following 


8 That is, we put an entry ((x;, T, Ø)) with a new variable x; between any two entries 
(Yi, 0i, Di)) and ((pi+1, 0:41, Di+1)) in o. 
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that position, i.e., ignoring possible intermediate inputs appearing later.’ 
The intuition is as above. For every recorded revision input, we assume one 
intermediate input which collects all the relevant intermediate inputs that 
have really occurred. 

If this claim is correct, we can introduce into o one entry (Xi, T, Ø) for ev- 
ery intermediate input. Thus we get a parametrised observation containing 
an entry for every revision input received. We can then construct a weakest 
acceptable core belief by instantiating each x; by x;, calculating the rational 
explanation of the observation thus obtained and then eliminating the addi- 
tional variables from the core belief. For example, given an observation o= 
((y1, 01, D1),-.--,(¢s5, 05, Ds)) and the information that intermediate inputs 
have been received only after ye and y4, we can calculate the weakest pos- 
sible core starting with o’ = ((~1, 91, D1), (2, 02, D2), (£1, T, Ø), (v2, T, Ø), 
(a3, T, Ø), (%3, 03, Ds), (ya, 04, Da), (£4, T, Ø), (%5, 05, Ds)) and eliminating 
the x; from Ay(o’). Again, all x; are propositional variables not contained 
in L(o). 

The above claim for limiting the number of assumed intermediate inputs 
follows almost immediately from the following proposition. 


Proposition 3.8. Let p = (Y1, .--, Yn) and o = (41, ..., Ym). Then there 
exists a o’ = (Y1, ..-, Yh) such that for alll <i<n 


FC (1,--+, Pi, A)) = flo’ (P1, ---; Pi, A))- 


Proof (Sketch). The proof of this result uses the fact that for every sequence 
o there is a logical chain o’ (a sequence of formulae where each formula is 
entailed by its successor) that behaves exactly like ø. That is f(o-p’) = 
f(o’ - p’) for all sequences p’. However, for this result it suffices that o and 
o’ behave equivalently for all prefixes of p. We then show that a suitable 
a’ exists, in fact using the rational explanation algorithm and hypothetical 
reasoning. Q.E.D. 


Note that this result is not trivial, as m can be (much) greater than 
n and in this case we have to find a shorter sequence yielding equivalent 
formulae for all 1 < i < n. This proposition tells us that we can replace 
one block of intermediate inputs ø by one of the proposed length and be 
guaranteed an equivalent formula being constructed in the calculation for 
each recorded revision input y; coming later in the observation. 

We want to remark that some care has to be taken when considering the 
general case, where several blocks of intermediate inputs exist. Then p in 
the proposition may contain more elements than just the recorded revision 

9 The above result—that one intermediate input between any two recorded ones is 


enough—is not applicable here. Intermediate inputs may not be allowed at every 
position. 
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inputs; it also contains intermediate ones. And thus we have to find a 
sequence o’ not of length n but j < n where j is the number of recorded 
inputs. We are currently investigating whether the number of intermediate 
inputs that have to be assumed can be reduced further without effect on 
the core belief calculated. 

Fixed number of intermediate inputs. If we are given a maximal (or 
exact) number n of intermediate inputs that may have occurred we can draw 
conclusions about the core belief of the agent using the following method. 
Due to Proposition 3.7 we should indeed assume the maximal number of 
intermediate inputs—n. So let o be the observation containing only recorded 
inputs. If o has less than n+2 recorded inputs and there are no restrictions 
as to the positions of the intermediate inputs, we can use the result that 
one intermediate input between any two recorded ones suffices to explain 
o; otherwise, there are not enough intermediate inputs for this result to be 
applicable. In this case, we create the set of all possible observations o’ 
where n intermediate inputs have been inserted in o: 


O' = {01 - (a1, T, Ø)) + 02+...+ (an, T, Ø)) + Ong, |O = 01+... + Ongt}- 


Here we have already replaced the unknown formulae by new variables. If 
we have information about the positions of the intermediate inputs we can 
also take this into account when constructing O’. The observation o; may be 
empty, so consecutive intermediate inputs are explicitly allowed. Now any 
possible core belief will entail V{4 | Cn(a) = Cn(ay(o’)) N L(o), 0’ € OF. 
Note that this formula itself need not be an o/-acceptable core, i.e., it may 
not really explain the observation using n intermediate inputs. Conclusions 
about beliefs and non-beliefs can only be safe if they are safe for every 
observation in O’. 


3.4 Summary 


In this section, we showed what can still be said about A if some of the com- 
pleteness assumptions about the observation o are weakened. We started 
by allowing unknown subformulae y; to appear in o. This can happen as 
the logical content of the revision inputs or the beliefs need not be com- 
pletely known. In case the observation still contains a record for every in- 
put received, the calculation of an optimal core belief is still possible. The 
proposed method for dealing with such parametrised observations was to 
instantiate the unknown subformulae x; with new variables and apply the 
rational explanation construction to the observation thus obtained. From 
this explanation we can safely conclude which beliefs must belong to the 
agents core belief no matter what the real instantiation of the x; was. 

We showed in [23] that although we can construct a core belief from 
L(o) this does not guarantee that o can be explained without extending the 
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language. The unknown subformulae may still have to contain variables 
not belonging to L(o). We claim that it is not useful to look for an optimal 
instantiation of the unknown subformulae. Weakest core belief and belief 
trace heavily depend on the choice of the instantiation of the x; and even 
if we had the correct ones, Section 2.4 showed that the conclusions drawn 
from the belief trace implied by our explanation are of limited use. Instead 
we argue that the x; should be instantiated with x; and reasoning be done 
based on the rational explanation. This allows us to draw correct conclu- 
sions about the actual core belief of the agent, which must entail the one 
calculated that way. Further, we can use hypothetical reasoning to verify 
other beliefs and non-beliefs (restricted to L(o)) implied by the explanation 
thus obtained. 

The additional assumption that the belief corresponding to the last re- 
vision input in the (parametrised) observation completely characterises A’s 
beliefs at that point once more need not help. It might not even convey 
additional information about the language of the agent’s epistemic state 
or of the unknown subformulae. Consider the parametrised observation 
((p A x, T, Ø), (=p, 7p, Z)). It might not be very interesting but it illus- 
trates the point. As ~p is inconsistent with the first input, x could be 
instantiated with any formula and still =p would completely characterise 
the agent’s final beliefs. 

We then further allowed intermediate inputs, i.e., the original observa- 
tion does not contain a record for every input received. Some observations 
can be explained only when assuming that intermediate inputs have oc- 
curred. When fixing their number and positions, the problem is reduced to 
partially known inputs. If the observing agent does not have this informa- 
tion, we sketched procedures for drawing conclusions about what A’s core 
belief must entail. 


4 Conclusion, Future and Related Work 


In this paper, we departed from the traditional belief revision setting of 
investigating what an agent should believe after receiving (a sequence of 
pieces of) new information in a given initial state. Instead, we place our- 
selves in the position of an observer trying to reason about another agent in 
the process of revising its beliefs. Coming up with models of other agents 
is useful in many application areas as informed decisions may improve the 
personal or group outcome of interactions. 

The basic and admittedly oversimplified setting we consider is that we 
are given an observation containing propositional information about the re- 
vision inputs received by an agent A and about its beliefs and non-beliefs 
following each input. We investigated several degrees of incompleteness of 
the information provided. From such an observation we try to get a clearer 
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picture of A. Assuming A to employ a particular belief revision frame- 
work, the general approach for reasoning about the agent is to “regress” 
the information contained in the observation to arrive at a possible initial 
state of the agent. This state completely determines the revision behaviour 
and therefore allows to draw conclusions about A’s beliefs at each point in 
time during the observation as well as future beliefs. Even under the very 
strict assumptions we impose, hardly any safe conclusions can be drawn. 
Intuitively, this is because coming up with A’s true initial state is virtually 
impossible. The observing agent can only try to extend and refine the ob- 
servation and reason hypothetically in the sense of testing conjectures about 
A in order to improve the model. 

It should be clear that the general question does not require the use of 
the belief revision framework we assumed. For future work, it might be 
interesting to see if similar results can be obtained when assuming A to 
employ a different framework. It would be interesting to see how differ- 
ent revision frameworks compare with respect to their power to explain an 
observation and whether there is a significant difference in the quality of 
the conclusions that can be drawn. Another important question is whether 
there is a way to actually find out which revision framework an observed 
agent employs or whether other assumptions can be verified. We claimed 
that it is not reasonable to look for the optimal instantiation of the un- 
known subformulae but rather do hypothetical reasoning restricted to L(o). 
However, in some applications it might be interesting to know what the 
actual revision input was that triggered a certain reaction in the agent. So 
comparing potential instantiations (possibly from a fixed set of potential 
formulae) could be a topic for future research. 

We want to remark that the methodology illustrated in this paper can 
also be applied in slightly modified settings. It is possible to construct an 
initial state that explains several observations in the sense that different 
revision sequences start in the same state. This is reasonable, e.g., when 
thinking about an expert reasoning about different cases (the initial state 
representing the expert’s background knowledge) or identical copies of soft- 
ware agents being exposed to different situations. Our work is focused on 
reasoning using observations of other agents, but observing oneself can be 
useful as well. By keeping an observation of itself an agent may reason 
about what other agents can conclude about it, which is important when 
trying to keep certain information secret. The results can also be applied 
for slight variations of the assumed belief revision framework. For example, 
it is possible to allow the core belief to be revised or to relax the restriction 
that new inputs are always appended to the end of p in an epistemic state 
|p, 4]. The interested reader is referred to [24]. 

Our work has contact points to many other fields in AI research. Most 
obvious is its relation to belief revision. The intuitive interpretation we 
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used for the assumed revision framework is incorporation of evidence [13]. 
However, the representation of the epistemic state as a sequence of formulae 
does not distinguish between background knowledge and evidence. When 
applying the results, a more detailed analysis of the intended meaning of the 
concepts involved and a corresponding interpretation of the results would 
be needed. Reasoning about other agents is central for many areas, e.g., 
multi-agent systems, user modelling, goal and plan recognition, etc. Here 
we investigated one specific aspect. In reasoning about action and change, 
the question is often to find an action sequence that would cause a particular 
evolution of the world—either to achieve some goal (planning), or to find 
out what happened (abduction). Often, the initial state and the effects of 
an action are specified. In our setting, the effect of a revision input is not 
quite clear. It might be accepted by the agent or not and beliefs triggered 
by the input heavily depend on the initial state. Trying to come up with 
hypotheses about the inner mechanisms of an observed system, which could 
be interpreted as its initial state that determines its future behaviour, is a 
topic treated also in induction. 

We are not aware of work that investigates reasoning about the evolution 
of an observed agent’s beliefs matching our setting. So we want to conclude 
by mentioning some papers that investigate similar questions. [11] considers 
a much richer belief revision framework in a dialogue context. However, the 
focus is on progressing beliefs through a sequence of speech acts starting in 
a given initial state of the agents. This and many other publications utilise 
modal logics for representing agents’ beliefs, [14] being another example 
also handling the dynamics of these beliefs. Often there are proof systems 
or model checkers for the logics presented, but model generation, which 
is what we are doing in this paper, generally seems to be a problem. This 
means that if the initial state is not given, hypotheses can only be tested (via 
proofs) but not systematically generated. However, this is what calculating 
a potential initial state and the corresponding belief trace is. 

The papers [2, 26], which are dealing with update rather than belief re- 
vision, start from a sequence of partial descriptions of an evolving world and 
try to identify preferred trajectories explaining this sequence. [2] intends to 
sharpen the information about the last state of the world and concentrates 
on a particular preference relation, giving a representation result. [26] com- 
pares different possible preference relations among trajectories, positioning 
the approach with respect to revision and update. However, both allow for 
arbitrary changes at any point in time, i.e., they do not allow to integrate 
information about which actions were performed nor reason about possible 
outcomes of an action. Recall that although our observation contains the 
revision input received, this does not mean that it is actually accepted. 
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Abstract 


We consider a logic for reasoning about composite strategies in games, 
where players’ strategies are like programs, composed structurally. 
These depend not only on conditions that hold at game positions 
but also on properties of other players’ strategies. We present an 
axiomatization for the logic and prove its completeness. 


1 Summary 


Extensive form turn-based games are trees whose nodes are game positions 
and branches represent moves of players. With each node is associated a 
player whose turn it is to move at that game position. A player’s strategy 
is then simply a subtree which contains a unique successor for every node 
where it is this player’s turn to make a move, and contains all successors 
(from the game tree) for nodes where other players make moves. Thus a 
strategy is an advice function that tells a player what move to play when the 
game reaches any specific position. In two-player win/loss games, analysis 
of the game amounts to seeing if either player has a winning strategy from 
any starting position, and if possible, synthesize such a winning strategy. 

In multi-player games where the outcomes are not merely winning and 
losing, the situation is less clear. Every player has a preference for certain 
outcomes and hence cooperation as well as conflict become strategically 
relevant. Moreover, each player has some expectations (and assumptions) 
about strategies adopted by other players, and fashions her response ap- 
propriately. In such situations, game theory tries to explain what rational 
players would do. 

In so-called small (normal form) games, where the game consists of a 
small fixed number of moves (often one move chosen independently by each 
player), strategies have little structure, and prediction of stable behaviour 
(equilibrium strategy profiles) is possible. However, this not only becomes 
difficult in games with richer structure and long sequences of moves, it is 
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also less clear how to postulate behaviour of rational players. Moreover, if 
we look to game theory not only for existence of equilibria but also advice 
to players on how to play, the structure of strategies followed by players 
becomes relevant. 

Even in games of perfect information, if the game structure is sufficiently 
rich, we need to re-examine the notion of strategy as a function that deter- 
mines a player’s move in every game position. Typically, the game position 
is itself only partially known, in terms of properties that the player can test 
for. Viewed in this light, strategies are like programs, built up systemati- 
cally from atomic decisions like if b then a where b is a condition checked 
by the player to hold (at some game position) and a is a move available to 
the player at that position. 

There is another dimension to strategies, namely that of responses to 
other players’ moves. The notion of each player independently deciding 
on a strategy needs to be re-examined as well. A player’s chosen strategy 
depends on the player’s perception of apparent strategies followed by other 
players. Even when opponents’ moves are visible, an opponent’s strategy 
is not known completely as a function. Therefore the player’s strategy is 
necessarily partial as well. 

The central idea of this paper is to suggest that it helps to study strate- 
gies given by their properties. Hence, assumptions about strategies can be 
partial, and these assumptions can in turn be structurally built into the 
specification of other strategies. This leads us to proposing a logical struc- 
ture for strategies, where we can reason with assertions of the form “(partial) 
strategy o ensures the (intermediate) condition a”. 

This allows us to look for induction principles which can be articulated in 
the logic. For instance, we can look at what conditions must be maintained 
locally (by one move) to influence an outcome eventually. Moreover, we can 
compare strategies in terms of what conditions they can enforce. 


The main contributions of this paper are: 


e We consider non-zero-sum games over finite graphs, and consider best 
response strategies (rather than winning strategies). 


e The reasoning carried out works explicitly with the structure of strate- 
gies rather than existence of strategies. 


e We present a logic with structured strategy specifications and formulas 
describe how strategies ensure outcomes. 


e We present an axiom system for the logic and prove that it is complete. 
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1.1 Other work 


Games are quite popularly used to solve certain decision problems in logic. 
Probably the best example of a logical game is the Ehrenfeucht-Fraissé game 
which is played on two structures to check whether a formula of a certain 
logic can distinguish between these structures [7]. Games are also used as 
tools to solve the satisfiability and model checking questions for various 
modal and temporal logics [12]. Here, an existential and a universal player 
play on a formula to decide if the formula is satisfiable. The satisfiability 
problem is then characterised by the question of whether the existential 
player has a winning strategy in the game. These kinds of games designed 
specifically for semantic evaluation are generally called logic games. 

Recently, the advent of computational tasks on the world-wide web and 
related security requirements have thrown up many game theoretic situa- 
tions. For example, signing contracts on the web requires interaction be- 
tween principals who do not know each other and typically distrust each 
other. Protocols of this kind which involve selfish agents can be easily 
viewed as strategic games of imperfect information. These are complex 
interactive processes which critically involve players reasoning about each 
others’ strategies to decide on how to act. In this approach, instead of de- 
signing games to solve specific logical tasks, one can use logical systems to 
study structure of games and to reason about them. 

Game logics are situated in this context, employing modal logics (in 
the style of logics of programs) to study logical structure present in games. 
Parikh’s work on propositional game logic [13] initiated the study of game 
structure using algebraic properties. Pauly [14] has built on this to pro- 
vide interesting relationships between programs and games, and to describe 
coalitions to achieve desired goals. Bonnano [5] suggested obtaining game 
theoretic solution concepts as characteristic formulas in modal logic. Van 
Benthem [2] uses dynamic logic to describe games as well as strategies. Van 
Ditmarsch [6] uses a dynamic epistemic language to study complex infor- 
mation change caused by actions in games. The relationship between games 
defined by game logics and that of logic games, is studied by van Benthem 
in [8]. 

On the other hand, the work on Alternating Temporal Logic [1] considers 
selective quantification over paths that are possible outcomes of games in 
which players and an environment alternate moves. Here, we talk of the 
existence of a strategy for a coalition of players to force an outcome. [8] 
draws parallels between these two lines of work, that of Pauly’s coalition 
logics and alternating temporal logic. It is to be noted that in these logics, 
the reasoning is about existence of strategies, and the strategies themselves 
do not figure in formulas. 
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In the work of [10] and [11], van der Hoek and co-authors develop logics 
for strategic reasoning and equilibrium concepts and this line of work is 
closest to ours in spirit. Our point of departure is in bringing logical struc- 
ture into strategies rather than treating strategies as atomic. In particular, 
the strategy specifications we use are partial (in the sense that a player may 
assume that an opponent plays a whenever p holds, without knowing under 
what conditions the opponent’s strategy picks another move b), allowing for 
more generality in reasoning. In the context of programs, logics like propo- 
sitional dynamic logic [9] explicitly analyse the structure of programs. This 
approach has been very useful in program verification. 


2 Game Arenas 


We begin with a description of game models on which formulas of the logic 
will be interpreted. We use the graphical model for extensive form turn- 
based multiplayer games, where at most one player gets to move at each 
game position. 


Game arena 

Let N = {1,2,...,n} be a non-empty finite set of players and © = {a1, a2, 
..+;Qm} be a finite set of action symbols, which represent moves of players. 
A game arena is a finite graph G = (W, —, wo, x) where W is the set 
of nodes which represents the game positions, — : (W xX) = W isa 
function also called the move function, wo is the initial node of the game. 

Let the set of successors of w € W be defined as w= {w! € W | w > w 
for some a € E}. A node w is said to be terminal if w= Ø. x: W > N 
assigns to each node w in W the player who “owns” w: that is, if x(w) = k 
and w is not terminal then player k has to pick a move at w. 

In an arena defined as above, the play of a game can be viewed as placing 
a token on wo. If player k owns the game position wo i.e x(wo) = k and 
she picks an action ’a’ which is enabled for her at wo, then the new game 
position moves the token to w’ where wọ —> w’. A play in the arena is 
simply a sequence of such moves. Formally, a play in G is a finite path 
p = wo = w, £> ... = wp where wp is terminal, or it is an infinite path 
p = Wo £, w, => ... where Vi: w; => wi+1 holds. Let Plays denote the 
set of all plays in the arena. 

With a game arena G = (W, —, wo, X), we can associate its tree unfold- 
ing also referred to as the extensive form game tree T = (S, =>, so, A) where 
(S, =>) is a countably infinite tree rooted at sọ with edges labelled by £ and 
A: S — W such that: 


e A(so) = Wo- 


e For all s,s’ € S, if s = s’ then A(s) > Xs’). 
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e If \(s) = w and w — w’ then there exists s’ € S such that s = s’ 
and X(s’) = w. 


Given the tree unfolding of a game arena J, a node s in it, we can define 
the restriction of T to s, denoted Ts to be the subtree obtained by retaining 
only the unique path from root sg to s and the subtree rooted at s. 


Games and winning conditions 


Let G be an arena as defined above. The arena merely defines the rules about 
how the game progresses and terminates. More interesting are winning 
conditions, which specify the game outcomes. We assume that each player 
has a preference relation over the set of plays. Let <'C (Plays x Plays) 
be a complete, reflexive, transitive binary relation denoting the preference 
relation of player i. Then the game G is given as, G = (G, {3 hen). 

Then a game is defined as the pair G = (G, (=')ien). 
Strategies 
For simplicity we will restrict ourselves to two player games, i.e., N = {1,2}. 
It is easy to extend the notions introduced here to the general case where 
we have n players. 

Let the game graph be represented by G = (W!, W?, —>, so) where W+ 
is the set of positions of player 1, W? that of player 2. Let W =W!UW?. 

Let T be the tree unfolding of the arena and sı a node in it. A strategy 
for player 1 at node sı is given by: u = (S}, S}, =p, 81) is a subtree of Ts, 
which contains the unique path from root so to sı in T and is the least 
subtree satisfying the following properties: 


e sı E€ S}, where x(X(s1)) = 1. 
e For every s in the subtree of TG rooted at s1, 


— if s € S} then for some a € X, for each s’ such that s => s’, we 
have s =>, 8’. 
—ifse Sis then for every b € X, for each s’ such that s A s', we 
have s 5, g 
Let Q; denote the set of all strategies of Player i in G, for i = 1,2. A 
strategy profile (4, T) defines a unique play p/, in the game G. 
3 The Logic 


We now present a logic for reasoning about composite strategies. The syntax 
of the logic is presented in two layers, that of strategy specification and game 
formulas. 
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Atomic strategy formulas specify, for a player, what conditions she tests 
for before making a move. Since these are intended to be bounded memory 
strategies, the conditions are stated as past time formulas of a simple tense 
logic. Composite strategy specifications are built from atomic ones using 
connectives (without negation). We crucially use an implication of the form: 
“if the opponent’s play conforms to a strategy m then play o”. 

Game formulas describe the game arena in a standard modal logic, and 
in addition specify the result of a player following a particular strategy at 
a game position, to choose a specific move a, to ensure an intermediate 
outcome a . Using these formulas one can specify how a strategy helps to 
eventually win an outcome a. 

Before we describe the logic and give its semantics, some prelimiaries 
will be useful. Below, for any countable set X, let Past(X) be a set of 
formulas given by the following syntax: 


y € Past( X) := x € X | aw | Y1 V Y2 | Ov. 


Such past formulas can be given meaning over finite sequences. Given 
any sequence € = toti---tm, V : {to tm} — 2%, and k such that 
0 < k < m, the truth of a past formula w € Past(X) at k, denoted £, k E w 
can be defined as follows: 


o £ k H piff pE V (tk). 

e ¿k H| y iff E€, k Fy. 

e £, k H Yi V be iff E, k H yi or £, k H Yoa. 

e £, k = Oy iff there exists a j :0 < j < k such that £, j = w. 


Strategy specifications 


For simplicity of presentation, we stick with two player games, where the 
players are Player 1 and Player 2. Let 7 = 2 when i = 1 and 7 = 1 when 
i= 2 

Let P’ = {pġ, pi, ...} be a countable set of proposition symbols where 
7 € P;, for i € {1,2}. Let P = PtU P? U {leaf}. 7 and T2 are intended 
to specify, at a game position, which player’s turn it is to move. The 
proposition leaf specifies whether the position is a terminal node. 

Further, the logic is parametrized by the finite alphabet set X = {a1, a2, 
.-:, Am} of players’ moves and we only consider game arenas over ©. 

Let Strat’ (PŻ), for i = 1,2 be the set of strategy specifications given by 
the following syntax: 


Strat’ (PŻ) := [Y = ak} | o1 +02 | 01- 02 | T > 0 


where r € Strat’ (P! N P?), y € Past(P*) and a, € È. 
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The idea is to use the above constructs to specify properties of strategies. 
For instance the interpretation of a player i specification [p +> a]* will be 
to choose move “a” for every i node where p holds. m = ø would say, at 
any node player 7 sticks to the specification given by ø if on the history of 
the play, all moves made by 7 conforms to 7. In strategies, this captures 
the aspect of players actions being responses to the opponents moves. As 
the opponents complete strategy is not available, the player makes a choice 
taking into account the apparent behaviour of the opponent on the history 
of play. 

For a game tree T, a node s and a strategy specification ø € Strat’ (PŻ), 
we define 7; No = (So, =v, 80) to be the least subtree of 7, which contains 
ps, (the unique path from so to s) and closed under the following condition. 


e For every s’ in So such that s = >* s’, 
. . a a 
— s' is an į node: s’ =} s” anda € o(s’) 6 9 =, 8”. 
. ii a a 
— s' is an 7 node: s = 8s" e 3! =>, 8”. 


Given a game tree 7 and a node s in it, let p$, : so A go SS 
Sm = s denote the unique path from so to s. For a strategy specification 
a € Strat’(P’) and a node s we define o(s) as follows: 


{a} ifs eW* and pfp m HY 
X otherwise 


lY = a]'(s) = [ 


e (o1 + 02)(s) = o1 (s) U o2(8). 


e (c1: 02)(s) = o1 (5) N o2(8). 


TEEN 
ane o(s) if Vy p <j <m, a; € T(sj) 
2 otherwise 


We say that a path ps s = 8, = s = Sm = 8’ in T conforms 
too if Vj: 1 <j <m, a; € o(s;). When the path constitutes a proper 
play, i.e., when s = sg, we say that the play conforms to ø. 

Syntax 
The syntax of the logic is given by: 


I :=pEP |-~a |a Va| (aja | G@a| Sa | (o)i:cla~; b 


where c € X, ø € Strat'(P’), B € Past(PŻ). The derived connectives ^, 
> and [aja are defined as usual. Let $a = =E~a, (N)a = Vacs (a)a, 
[N]a = ~/(N})-a, (P)a = Vacs (@)a and [P] = 7(P)na. 


aed 
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The formula (ø); : c asserts, at any game position, that the strategy 
specification o for player i suggests that the move c can be played at that 
position. The formula o ~; 2 says that from this position, there is a way of 
following the strategy o for player 7 so as to ensure the outcome 3. These 
two modalities constitute the main constructs of our logic. 


Semantics 

The models for the logic are extensive form game trees along with a valua- 
tion function. A model M = (T,V) where T = (S', S?, —, so) is a game 
tree as defined in Section 2, and V : S — 2? is the valuation function, such 
that: 


e leaf € V(s) iff moves(s) = y. 


where for any node s, moves(s) = {a | s => s’}. 
The truth of a formula a € II in a model M and position s (denoted 
M, s = a) is defined by induction on the structure of a, as usual. Let p3, 


ao am-1 
be so => s1 = > Sm =S. 


e M,sEpiff pe V(s). 


e M,s =| ~a iff M,s Fa. 


e M,s =| a V ag iff M,s — a, or M,s = ag. 


e M,s (aja iff there exists s’ € W such that ss’ and M,s’ Ea. 


e M,s H (@)a if m > 0, a = am-ı and M, Sm-1 Ea. 


e M,s — a iff there exists j : 0 < j < m such that M, sj = a. 


e M,s H (o);: cif cE o(s). 


e M,s Ko ~; B iff for all s’ in 7; ) o such that s —* s', we have 
M, s’ H BA (Ti > enabled,), 


where enabled, = V ,cy,((a) True A (0); : a). 

The notions of satisfiablility and validity can be defined in the standard 
way. A formula a is satisfiable iff there exists a model M, there exists s 
such that M,s = a. A formula a is said to be valid iff for all models M, 
for all s, we have M, s Fa. 
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4 Example 


Probably the best way to illustrate the notion of strategy specification is to 
look at heuristics used in large games like chess, go, checkers, etc. A heuristic 
strategy is basically a partial specification, since it involves checking local 
properties like patterns on the board and specifying actions when certain 
conditions hold. For instance, a typical strategy specification for chess would 
be of the form: 


e If a pawn double attack is possible then play the action resulting in 
the fork. 


Note that the above specification is in contrast with a specific advice of the 
form: 


e If a pawn is on f2 and the opponent rook and knight are on e5 and g5 
respectively then move f2-f4. 


A strategy would prescribe such specific advice rather than a generic one 
based on abstract game position properties. Heuristics are usually employed 
when the game graph being analysed is too huge for a functional strategy 
to be specified. However, we refrain from analysing chess here due to the 
difficulty in formally presenting the game arena and the fact that it fails 
to give much insight into the working of our logic. Below we look at a few 
simple examples which illustrates the logic. 


Example 4.1. Consider the game shown in Figure 1. Players alternate 
moves with 1 starting at so. There are two cycles C1 : 855 > S6 > 87 > 
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Sg — 85, Co: 81 S2 83 S4 sı and two terminal nodes tı and tg. 
Let the preference ordering of player 1 be tı <! tg <! Cy <! Cı. As far 
as player 2 is concerned tı <? Cı and he is indifferent between Cz and tə. 
However, he prefers C2 or tz over {C1, tı}. Equilibrium reasoning will advise 
player 1 to choose the action “b” at so since at position s7 it is irrational for 
2 to move « as it will result in 2’s worst outcome. However the utility differ- 
ence between C1 and tı for 2 might be negligible compared to the incentive 
of staying in the “left” path. Therefore 2 might decide to punish 1 for mov- 
ing b when 1 knew that {C2,t2} was equally preferred by 2. Even though 
tı is the worst outcome, at s7 player 2 can play x to implement the pun- 
ishment. Let V(p;) = {s3, 57}, V (Pint) = {50}, V (Pgood) = {S0, $1, $2, 83, S4} 
and V(ppunish) = {S0, S5, S6, $7,ti}. The local objective of 2 will be to re- 
main on the good path or to implement the punishment. Player 2 strategy 
specification can be written as 


T = ([pinit > b] => [pj > x]*) - ([pinit > a] > [pj > yl’). 


We get that m ~*2 (Pgood V Punish). Player 1, if he knows 2’s strategy, might 
be tempted to play “a” at sq by which the play will end up in C2. Let the 
proposition Pworst hold at tı which is the worst outcome for player 1. Then 
we have [Pint > a]! ~+1 7Pworst- This says that if player 1 chooses a at the 
initial position then he can ensure that the worst outcome is avoided. 


A G AG og 
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| a K | = —) 
a2 IN a IN 
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c°? ce? 
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FIGURE 2. Sabotage Game. 


Example 4.2. The sabotage game [4] is a two player zero sum game where 
one player moves along the edges of a labelled graph and the other player 
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removes an edge in each round. Formally let a X labelled graph R for some 
alphabet set © is R = (V, e) where V is the set of vertices ande: Vx X — V 
is the edge function. The sabotage game is played as follows: initially 
we consider the graph Ro = (Vo, eo, vo). There are two players, Runner 
and Blocker who move alternatingly where the Runner starts the run from 
vertex vo. In round n, the Runner moves one step further along an existing 
edge of the graph. I.e., he chooses a vertex uni, € V such that there 
exists some a € È with en(Un, a) = Un4i. Afterwards the Blocker removes 
one edge of the graph. i.e., he chooses two vertices u and v such that 
for some a € ÈX, e,(u,a) = v and defines the edge function e,+1 to be 
same as that of e, except that e,+1(u,a@) will not be defined. The graph 
Rn+1 = (V,en41;Un+1). We can have a reachability condition as the winning 
condition. I.e., the Runner wins iff he can reach a given vertex called the 
goal. The game ends, if either the Runner gets stuck or if the winning 
condition is satisfied. 

It is easy to build a conventional game arena for the sabotage game 
where player positions alternate. The game arena will have as its local 
states subgraphs of R with the current position of Runner indicated. I.e., 
W = Edges x V where Edges is the set of all partial edge functions e : 
V x5 — V. Let Wt and W? be the set of game positions for Runner and 
Blocker respectively. The initial vertex so = (eo, vo) and so € W!. Let 
s =(e,v) and s’ = (e’,v’) be any two nodes in the arena. The transition is 
defined as follows. 


e ifs € W! and e(v,a) =v’ then s > s’, e = e' and s' € W? 


e if s € W?, for some u, u’ € W we have e(u,a) = u’ and e’ is the same 
( ‘) 


as e except that e/(u,a) is not defined, then s “225 s’, v = v’ and 


sew. 


Figure 2 shows the first three game positions in a possible run of the 
sabotage game. The game starts with Runner moving from node A to node 
B. The blocker then removes the edge az adjacent to node B and the game 
continues. In the formulas given below, we will refer to Runner as player 1 
and Blocker as player 2. 

Since the Runner’s objective is to not get stuck, he might reason as 
follows. If it is the case that the Blocker always removes an edge adjacent 
to the node that Runner has currently selected then try to move to a node 
which has multiple outgoing edges. We use the following propositions: 


e present,,: denotes that the current node of runner is v 


e adj„: denotes that the adjacent node has multiple edges 
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Let r, denote the action which removes an adjacent edge of v and moveag; 
denote the action which moves to the adjacent node with multiple edges. 
The Runner’s specification can be given as: 


e [present, +> ry]? => [adj > moveagi]' 


Consider the situation where all the nodes in the graph have a single 
outgoing edge and the goal state is a single state. It is quite easy to show 
that in such a game, the Runner wins iff the start node is the goal or if there 
is an edge connecting the start node with the goal. This property can be 
captured by the following proposition: 


e gB: denotes that in the graph the start node is not the goal and 
there is no single edge between start and goal nodes. In other words 
the graph is “nice” for Blocker. 


° adj: denotes that the Runner’s current node is one adjacent to the 
goal node. 


Let rfa denote the action which removes the edge connecting the current 
node of Runner with the goal. Consider the following formula: 


: CHA A adj7) m rial wD (leaf 2 win) 


This says that if the graph is “nice” for Blocker and if the current se- 
lected node of Runner is one adjacent to the goal then remove the only edge 
connecting it with the goal. In all the other cases the Blocker can remove 
any random edge, and so this need not be mentioned in the strategy speci- 
fication. This specification ensures that when the terminal node is reached 
then it is winning for Blocker. 


5 Axiom System 


We now present our axiomatization of the valid formulas of the logic. Before 
we present the axiomatization, we will find some abbreviations useful: 


e root = —(P)True defines the root node to be one that has no prede- 
cessors. 


i 


e (a) = Ti A (a); : a denotes that move “a” is enabled by ø at an i 
node. 
e inv? (a, 8) = (Ti^ (0); : a) > [a](o ~; B) denotes the fact that after an 
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a” move by player 7 which conforms to ø, o ~; B continues to hold. 


e inve(3) = m > [N](o ~; B) says that after any move of 7, o ~; B 
continues to hold. 


e conf, = El((@)t% > (@)(m)z: a) denotes that all opponent moves in the 
past conform to 7. 
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The axiom schemes 
(AO) All the substitutional instances of the tautologies of propositional cal- 


culus. 
(A1) (a) [a](a1 > a2) > (la]aı > [alaz) 
(b) [a (ai > a2) > ([@ja1 > [alaz) 


(A2) (a) (aja > [ala 

(b) (@)a > [aja 

(c) (@) True > —~(b) True for all b 4a 
(A3) (a) a> ja] @a 

(b) a > fā] laja 
(A4) (a) © root 


Fla = (a A [P] Ea) 


(A5) 


(A6) 


(A7) o ~; B > (BA inv? (a, B) A inv (8) A (leaf > enabled,)) 


Inference rules 


a, adZBp a a 
oe. (MP NG NG- 
so (uP) 5 (we) 5 (NG) 
2 > [Pleo (Ind-past) 
a> Ha 
aA ð; (a) > lala, aA > [Na, 
a / leaf > enabled,, a> 8 iinde 


a Doai B 


The axioms are mostly standard. After the Kripke axioms for the (a) 
modalities, we have axioms that ensure determinacy of both (a) and (@) 
modalities, and an axiom to assert the uniqueness of the latter. We then 
have axioms that relate the previous and next modalities with each other, as 
well as to assert that the past modality steps through the (@) modality. An 
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axiom asserts the existence of the root in the past. The rest of the axioms 
describe the semantics of strategy specifications. 

The rule Ind-past is standard, while Ind ~ illustrates the new kind of 
reasoning in the logic. It says that to infer that the formula o ~; 8 holds 
in all reachable states, @ must hold at the asserted state and 


e for a player i node after every move which conforms to øg, 3 continues 
to hold. 


e for a player 7 node after every enabled move, 8 continues to hold. 
e player į does not get stuck by playing ø. 


To see the soundness of (A7), suppose it is not valid. Then there exists 
a node s such that M, s = o ~; @ and one of the following holds: 


e M,s Æ 8: In this case, from semantics we get that M,s Fa~; B 
which is a contradiction. 


e M,s inv? (a, 3): In this case, we have s € W*, M,s H (o); : a and 
M,s' Ko ~; B where s => s’. This implies that there is a path p2} 
which conforms to ø and either M, sp | B or moves(sk) N ao(sk) = ¢. 
But since s => s’, we have p$r conforms to ø as well. From which it 
follows that M, s Æ o ~; B which is a contradiction. 


e M,s inv2(@): We have a similar argument as above. 


e M,s (A leaf > enabled: This means that M,s = —leaf and M,s |- 
enabled,. Therefore moves(s) N o(s) = Ø and by semantics we have 
M,s Æ o ~; B which is a contradiction. 


To show that the induction rule preserves validity, suppose that the 
premise is valid and the conclusion is not. Then for some node s we have 
M,s = a and M,s Fo ~; p. i.e., there is a path p$” which conforms to 
a such that M, sk A @ or sp is a non-leaf node and o(s) N moves(s;,) = 9. 
Let p$r be the shortest of such paths. 

Suppose M, sk Æ B, then we have the following two cases to consider. 


e sk-ı E€ W*: By assumption on the path p%*, we have M,sķ-1 H 
ao? (ag—1). From validity of a > (the premise), we have M, sz E a, 
which implies M, sk-1 | [ax—-iJa. Therefore we get M,sp-1 Æ (aA 
ô? (ak—1)) > [ax—1]@, which gives us a contradiction to the validity of 
a premise. 


e s~—1 E€ W": By assumption on the path p%*, we have M, sp_1 = QAT. 
Using an argument similar to the previous case we also get M, sk-1 |K 
[ax—i1Ja. Therefore we have M,s,-1 JF (a A 77) > [N]a, giving us a 
contradiction to the validity of a premise. 


A Logical Structure for Strategies 197 


If sx is a non-leaf node and o(s;) Mmoves(s;) = y then we have M, sk H 
a/ leaf and M, sk /K enabled,. Therefore M, sk £ (a A aleaf) > enabled,, 
which is the required contradiction. 


6 Completeness 


To show completeness, we prove that every consistent formula is satisfiable. 
Let ap be a consistent formula, and let W denote the set of all maximal 
consistent sets (MCS). We use w,w’ to range over MCS’s. Since ao is 
consistent, there exists an MCS wo such that ap € wo. 

Define a transition relation on MCS’s as follows: w > w' iff {(a)ala € 
w} C w. We will find it useful to work not only with MCS’s, but also 
with sets of subformulas of ao. For a formula a let CL(a) denote the 
subformula closure of a. In addition to the usual downward closure, we 
also require that ©root, leaf € CL(a) and o ~; B € CL(a) implies that 
B, inv? (a, 6), inv? (8), enabled, € CL(a). Let AT denote the set of all max- 
imal consistent subsets of CL(ag), reffered to as atoms. Each t € AT isa 
finite set of formulas, we denote the conjunction of all formulas in ¢ by t. 
For a nonempty subset X C AT, we denote by X the disjunction of all 
t,t € X. Define a transition relation on AT as follows: t > t iff tA (a yi 
is consistent. Call an atom t a root atom if there does not exist any atom t’ 
such that t —*> t for some a. Note that to = wo N CL(ao) € AT. 


Proposition 6.1. There exist t1,...,t, € AT and aj,...a, E€ È (k > 0) 
such that tgp => tp_1... => to, where tp is a root atom. 


Proof. Consider the least set R containing to and closed under the following 
condition: if tı € R and for some a € 5 there exists tz such that t2 —> ty, 
then t2 € R. Now, if there exists an atom t’ € R such that t is a root then 
we are done. Suppose not, then we have H R>-root. But then we can 
show that + R > [P]R. By rule Ind-past and above we get + R > Ehroot. 
But then to € R and hence H f > R and therefore we get F to > Eroot, 
contradicting axiom (A4a). Q.E.D. 


Above, we have additional properties: for any formula Ga € tk, we also 
have a € ty. Further, for all j € {0,--- , k}, if Sa € tj, then there exists i 
such that k > i > j anda €E ti. Both these properties are ensured by axiom 
(A4b). A detailed proof can be found in the Appendix, Lemma A.2. 

Hence it is easy to see that there exist MCS’s wi, .-., Wk E W and 
a1,..-ap E€ © (k > 0) such that wk L5 we_1-.. => wo, where w; N 
CL(ao) = tj. Now this path defines a (finite) tree Ta = = (So, 0, S0) rooted 
at so, where Sp = {So, 51,..., Sk}, and for all j € {0,--- ,k}, s; is labelled 
by the MCS wr_;. The relation =o is defined in the obvious manner. 
From now we will simply say a € s where s is the tree node, to mean that 
a E€ w where w is the MCS associated with node s. 
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Inductively assume that we have a tree Ty = (Sk, =k, 80) such that 
the past formulas at every node have “witnesses” as above. Pick a node 
s € Sp such that (a)True € s but there is no s’ € Sp such that s 5 9’. 
Now, if w is the MCS associated with node s, there exists an MCS w such 
that w —> w’. Pick a new node s’ ¢ Sk and define Tk+1 = Sk U {s’} and 
=r = = U{(s,a,5’)}, where w’ is the MCS associated with s’. It is 
easy to see that every node in Tk+}1 has witnesses for past formulas as well. 

Now consider T = (S, =, so) defined by: S = U,zs) Sk and => = 
Un>o =>k- Define the model M = (T, V) where V(s) = wN P, where w is 
the MCS associated with s. 


Lemma 6.2. For any s € S, we have the following properties. 
1. if [aja € s and s => s’ then a € s’. 
2. if (aja € s then there exists s’ such that s = s’ and a € 8’. 
3. if [aja € s and s = s then a € s’. 


4. if (@)a E s then there exists s’ such that s’ => s and a € 8’. 


5. if Fa € s and s’ =* s then a € 3’. 


6. if Ga € s then there exists s’ such that s’ —>* s and a € 8’. 


Proof. Cases (1) to (5) can be shown using standard modal logic techniques. 
Case (6) follows from the existence of a root atom (Proposition 6.1) and 
axiom (A4b). Q.E.D. 


Lemma 6.3. For all y € Past(P), for all s € S, w € s iff ps, s Ev. 


Proof. This follows from Lemma 6.2 using an inductive argument. Q.E.D. 


Lemma 6.4. For all i, for all o € Strat’ (Pt), for all c € X, for all s € S, 
(a)i: c€ siff c € o(s). 


Proof. The proof is by induction on the structure of ø. The nontrivial cases 
are as follows: 
a= (wr al: 


(=) Suppose ([v + a]'); : c € s. If c = a then the claim holds trivially. 
If c Æ a then from (A5a) we get that aw € s, from Lemma 6.3 ps,s E Y. 
Therefore by definition we have [y + a]'(s) = £ and c € o(w). 


(<=) Conversely, suppose (fp > alt); : c ¢ s. From (Ada) we have a Æ c. 
From (A5b) we get w € s. By Lemma 6.3 p,,s H Y. Therefore c ¢ o(s) by 
definition. 
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o =T => 0°: Let p$, : 80 oo, .., SS sp = s be the unique path from the 


root to s. 


(=) Suppose (m = ao’); : c € s. To show c € (t => 0’)(s). Suffices to show 
that p$, conforms to 7 implies c € o’(s). From (A6c) we have conf, > (a); : 
c € s. Rewriting this we get O((@)7 A [@|(A(7)z : a)) V (0’)i : c € s. We 
have two cases, 


e if (o’); : c € s then by induction hypothesis we get c € o’(s). Therefore 
by definition c € (7 > a);(s). 


e otherwise we have ©((@)7; A [@](-=(7)z: a)) € s. From Lemma 6.2(6), 
there exists s; E€ ps such that (@)7; A [@|(A(7)z : a) € sı. By Lemma 
6.2(4) there exists s;-1 E€ ps NW” such that s;_1 => sı. From Lemma 
6.2(3), a(m)z : a € 8,1. Since s;_1 is an MCS, we have (m)z: a € sı—1. 
By induction hypothesis, a ¢ 7(s;—1), therefore we have p$, does not 
conform to 7. 


(<) Conversely, using (A6c) and a similar argument it can be shown that 
if (m > 0’);:¢€¢sthenc¢ (r > 0')(s). Q.E.D. 


Theorem 6.5. For all a € II, for all s € S, aœ € s if M, s Ea. 


Proof. The proof is by induction on the structure of a. a = (ø); : c. From 
Lemma 6.4 we have (a); : c € s iff c € o(s) iff by semantics M, s — (0); : c. 


a =o ~i pb. 
(=) We show the following: 


(1) If o ~; 6 € s and there exits a transition s => s’ such that a € o(s), 
then {3,0 ~; 3} C s’. Suppose o ~; B € s, from (AT) we have 8 € s. 
We have two cases to consider. 


e s € Wt: We have 7; € s. Since a € o(s), by Lemma 6.4 we have 
(a); : a € s. From (A7) we get |a](o ~; 8) € s. By Lemma 
6.2(1) we have o ~; B E S. 

e s € W': We have m € s. From (A7) we get [N](o ~; 8) € s5, 
since s is an MCS we have for every a € X, [a](o ~; 3) € s. By 
Lemma 6.2(1) we have o ~; B € 3’. 


By applying (A7) at s’ we get 8 € s’. 


(2) If o ~; B € s and s is a non-leaf node, then Js’ such that s = s’ 
and a € g(s). 


Suppose s is a non-leaf node. From (A7), Vacgz((a) True^ (o); : a) € s. 
Since s is an MCS, there exists an a such that (a)True A (o); : a € s. 
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By Lemma 6.2(2), there exists an s’ such that s => s’ and by Lemma 
6.4 a € o(s). 


(1) ensures that whenever o ~; 3 € s and there exists a path pĝ which 
conforms to a, then we have {3,0 ~i B} C sp. Since 8 € Past(P), by 
Lemma 6.3 we have M,s;, H 8. (2) ensures that for all paths p3* which 
conforms to a, if s is a non-leaf node, then moves(s)N o(s) 4 p. Therefore 
we get M, s H| o ~i p. 


(<) Conversely suppose o ~; B ¢ s, to show M, s Ka ~; B. Suffices to 
show that there exists a path p$* that conforms to ø such that M, sk E 6 
or sz is a non-leaf node and moves(sp) N o(s) = Y. 


Lemma 6.6. For all t € AT, o ~; B Z t implies there exists a path 
pe stat, Sar te... Œ- az ty which conforms to ø such that one of 


the following conditions hold. 
o BEtr. 


e tą is a non-leaf node and moves(t,) N (tk) = y. 


We have t = s N CL(o ~; B) is an atom. By Lemma 6.6 (proof given 
in the Appendix), there exists a path in the atom graph t = tı Sar 
to... “Sar tk such that 6 ¢ ty or ty is a non-leaf node and moves(t,) N 
o(tk) = p. tı can be extended to the MCS s. Let t3 = t2 U {allaiJa € s}. 
Its easy to check that t is consistent. Consider any MCS s2 extending th, 
we have s => 82. Continuing in this manner we get a path in s = sı = 
S2... = Sk in M which conforms to o where either 8 Z Sk or Sk is a 
non-leaf node and moves(s%) N o(s) = 4. Q.E.D. 


7 Extensions for Strategy Specification 


Until operator 
One of the natural extensions to strategy specification is to come up with a 
construct which asserts that a player strategy conforms to some specification 
o until a certain condition holds. Once the condition is fulfilled, he is free 
to choose any action. 

We can add the future modality Oa in the logic defined in Section 3 
with the following interpretation. 


e M,s H Oy iff there exists an s’ such that s =>* s’ and M,s’ = 4. 


Let Past(II) and Future(II) denote the past and future fragment of II 
respectively. I.e., 


Past(II1”") := p E€ P’ | ~a | a, V a2 | a 
Future(II”’ ) = p E€ P’ | ~a | a1 V ag | Ca 
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Let Oa = ~~a and Ha = ©-a. We can enrich Strat’(P’) with the 
until operator Uy, where y € Past(II”’)UFuture(II”’ ), with the following 
interpretation: 


© (oUy)(s) = 


Note that until does not guarantee that y will eventually hold. We can 
extend the axiomatization quite easily to handle the new construct. Firstly 
we need to add the following axiom and the derivation rule for the future 
modality. 


> if 4j:0<7<m such that p51, j H 


a(s) otherwise 


| 
aS) 


a = (a A |N]Oa) (Ax-box) 
eMe (Ind) 


Using the above axiom and inference rule one can easily show the ana- 
logue of Lemma 6.2 and Lemma 6.3 for the future modality. For the until 
operator we have the following axiom. 


(cUy);: c= 79 D (oji: € (Ax- Until) 
We can show that Lemma 6.4 holds once again, for the extended syntax: 
Lemma 7.1. For all i, for all ø € Strat’(P"), for all c € Ð, for all s € S, 
(0); :¢€ siff cE o(s). 


Proof. The proof is by induction on the structure of ø as seen before. The 
interesting case is when o = d'Ugo: 


(=) Suppose (o/Uy); : c € s. It suffices to show that Vj : 0 < j < k, 
p3, j Ky implies c € o’(s). From axiom (Ax-Until), we have =Oy > (a’); : 
cE s. Rewriting this, we get Op € s or (0’); : CE s. 


e if Op € s, then by Lemma 6.2, Jj : 0 < j < k such that y € sj. 
Therefore we have ps), H y. 


e if (7); : c € s, then by induction hypothesis we have c € o(s). 


(<) To show (o’Uy); : c é s implies c ¢ (o’Uy)(s). It suffices to show that 
Vj: 0< 5 <m, psi,j jọ and c ¢ o'(s). From axiom (Ax-Until), we have 


ny A 7((0'); : c) E€ s. Rewriting this we get Fay € s and 7((0); : c) € s. 


e Ely € s implies Yj : 0 < j < m, ~y € s;(by Lemma 6.2). Since s; is 
an MCS, a ¢ sj. Therefore we have Yj :0 < j < m, psi, j Ky. 


~((o);i : c) € s implies (a); : c ¢ s(Since s is an MCS). By induction 
hypothesis we have c ¢ o(s). 


202 R. Ramanujam, S. Simon 


Nested strategy specification 

Instead of considering simple past time formulas as conditions to be verified 
before deciding on a move, we can enrich the structure to assert the oppo- 
nents conformance to some strategy specification in the history of the play. 
This can be achieved by allowing nesting of strategy specification. We can 
extend the strategy specification syntax to include nesting as follows. 


M:=¢lo|mAre 


Strat’,.(P*) := [y al’ | o1 +02 | 01-02 


where 7 € Past(P’), o € Strat’,.(P’) and y € I’. Below we give the 
semantics for the part that requires change. For the game tree 7 and a 


node s in it, let p$, : so Ss s1- = sm = s denote the unique path from 
So to s 
a if sc Wt and pê ,m 
e [y= ali(s) = Ai 
X otherwise 


e p mH oif Vj:0< j< m, aj € ø(si). 


bd Por M F 71 A Ya iff Pio M YI and Pio M = y2. 


For a past formula w, the notion of p$, m — w is already defined in 
Section 3. Let L denote the logic introduced in Section 3 and Lye. be 
the same as L except that o € Strat',.(P’). We show that L and Lyec 
have equivalent expressive power. Therefore one can stick to the relatively 
simple strategy specification syntax given in Section 3 rather than taking 
into account explicit nesting. 

It is easy to see that any formula y € I’, can be rewritten in the form 
oa’ Aw where o’ € Strat’,.(P*) and y € Past(P"). This is due to the fact that 
if Y1, Y2 € Past(P*) then Y1 A we € Past(P*) and c1 A 02 = 01 + o2(formally 
Vs, psp m Foi A o2 iff p$ m For: o2). 

Given Grec € Strat’..(P’) the equivalent formula o € Strat’ (PŻ) is con- 
structed inductively as follows. 
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Proof. The proof is by induction on the structure of ø. Let s € S and 
Piso £ 80 5 5,--- = sm = s be the unique path from root to s. 


oO 


[y => a]: Follows from the definition. 
o =0 1:02 and o = 01 + 02 follows easily by applying induction hypothesis. 


o = [n al): We need to show that for all s, [7 + a] (s) = ([r] => [True 
a|)(s). We have the following two cases: 


e p$ mMm} 7: In this case, we have [7 +> a](s) = a. p§,,m H= m implies 
Vj:0<j <m, a; € 7(s;). From induction hypothesis, a; € [7](s;), 
which implies p$, conforms to [7]. From the semantics, we get ([7] = 
[True +> a])(s) = ([True > a])(s) = a. 


e p$ m A T: In this case, we have [r +> a](s) = X and 4j:0< 7 <m 
such that a; ¢ (sj). By induction hypothesis, we have a; ¢ [7](s;) 
which implies that p$, does not conform to [r]. From semantics we 
get that ([7] = [True => a])(s) = X. 


o = [r Awa: The following two cases arise: 


e p.m = TAY: We have [r AY al(s) =a. p$, m H T Ay implies 
Pom = r and pfp m H Y. pm H m implies Vj : 0 < j < m, 
aj € m(s;). By induction hypothesis, a; € [m] (sj) and as before we 


get ([7] = [v > a])(s) = (b> a])(s) = a. 


e p$ m Aa: We have the following two cases: 


— P$, M jÆ Y: It is easy to see that 


mAb al(s) = ([r] > W > a))(s) = X. 


— p$ mMm Æ m: In this case, Jj : 0 < j < m such that a; ¢ n(s;). 
By induction hypothesis, we have a; ¢ [r] (s;). By an argument 
similar to the one above we get [r] > [y  a])(s) = È. 


Q.E.D. 


For the converse, given a ø € Strat’(P’), we can construct an equiva- 
lent formula oye. € Strat,..(P’). The crucial observation is the following 
equivalences in Strat’ (PŻ). 


© T >01 +02 = (T > 01) + (7 > 02) 


© T> 01:02 = (T > 01): (T > 02) 
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© Tı > (m2 > 0) = (T1 - T2) > 0 


Using the above equivalences, we can write the strategy specification o 
in a normal form where all the implications are of the form 7 > [Y > a]. 
Then Grec is constructed inductively as follows: 


[iy al] = [br a] 
loi + 92] = [oi] + [2] 
[o1 - o2] = [or] - [e2] 

[ 


[7 > [y > al] = [fr] ^Y => a] 
Lemma 7.3. For all i, for all s € S, for all o € Strat’ (PŻ), o(s) = [o](s). 


Proof. The proof is by induction on the structure of formula. Let p$, : 
so = s1 > sm = s. The interesting case is when o = 7 > [y > al. 
We need to show that for all s, 7 > [w+ aļ(s) = [t A Wt al(s). We have 
the following two cases: 


e p$, conform to m: We have t > [i +> aļ(s) = [Yy |> a](s) and Yj : 
0< j <m, aj € T(si). By induction hypothesis, a; € [m] (s;) which 
implies that p3,,m = m. Therefore |[r] Ay = a](s) = [Yy = a](s). 


e p;, does not conform to 7: By an argument similar to the above, we 
can show that 7 > [y => al(s) = [r A 4 | aļ(s) = È. 


Q.E.D. 
Theorem 7.4. Logics L and Lyec have equivalent expressive power. I.e., 


e For every a € II, there exists Qyee € Irec such that M,s H a iff 
M, SF Qrec- 


e For every Qrec € Irec there exists a € II such that M,s E avec iff 
M,s =a. 


Proof. The theorem follows from Lemma 7.2 and Lemma 7.3 by a routine 
inductive argument. Q.E.D. 


8 Discussion 


We have defined a logic for reasoning about composite strategies in games. 
We have presented an axiomatization for the logic and shown its complete- 
ness. 

We again remark that the presentation has been given for two-player 
games only for easy readability. It can be checked that all the definitions 
and arguments given here can be appropriately generalized for n-player 
games. 
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While our emphasis in the paper has been on advocating syntactically 
constructed strategies, we make no claims to having the “right” set of con- 
nectives for building them. This will have to be decided by experience, 
gained by specifying several kinds of strategies which turn out to be of use 
in reasoning about games. 

We believe that a framework of this sort will prove useful in reasoning 
about multi-stage and repeated games, where strategy revision based on 
learning other players’ strategies (perhaps partially) plays an important 
role. 


Appendix 


Lemma A.1. For atoms tı and t2, the following statements are equivalent. 
1. f A (a)t is consistent. 
2. mh A tg is consistent. 


Proof. Suppose (@)f ^É is consistent, from (A3b) we have (a) 4 A [@](a) ta is 
consistent. Therefore, (@) (& A lajt) is consistent, which implies  {a]7(t1 ^ 
lajt). From (NG-), 7 =(t; A (a)tz), thus we have that f1 A (a)t2 is consistent. 

Suppose f; A (a) tz is consistent, from (A3a) we have [a] (@)fi A (a) ta is con- 
sistent. Therefore, (a) ((@ yea / Ata) is consistent, which implies 7 [a]>((@ Vt A 
ta). From (NG-),  7((a)é A f2), thus we get that (@)t, A ty is consis- 
tent. Q.E.D. 


Lemma A.2. Consider the path tp => t,_1... => to where tp is a root 
atom. 


1. For all j € {0,...,k — 1}, if [ala € tj and tj41 —> ty then a € ty41. 
2. For all j € {0,...,k — 1}, if (apa € tj and tia B tj then b = a and 
wE tj+1- 


3. For all j € {0,...,k — 1}, if Oa € tj then there exists i: j <i < k 
such that a € ti. 


Proof. 


(1) Since tj+1 5 tj, we have A A(a E is consistent, By Lemma A.1, tj ^ 
(@) t+ is consistent, which implies [a aja A (@)t;+1 is consistent (by omitting 
some conjuncts). Therefore (@)(a ^ i) is consistent. Using (NG-) we get 
aN baa is consistent and since ¢;+1 is an atom, we have a € tj41. 


(2) eas ty ans tj, we first show that b = a. Suppose this is not true, 


since tj41 = tj, we have t; \(b)t;11 is consistent. And therefore í; A (b) True 
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is consistent. From axiom (A2c) f; A [@]False is consistent. If (@)a € tj, then 
we get (@)a A [a]False is consistent: Therefore (@)(a A^ False) is consistent. 
From (NG-) we have a A False is consistent, which is a contradiction. 

To show a € tj+ı observe that (@a € t; implies [aja € t; (by axiom 
(A2b) and closure condition). By previous argument we get a € tj41. 


(3) Suppose a € tj and tj41 —> tj. If a € tj then we are done. Else by 
axiom (A4b) and the previous argument, we have (@) a €E tj. From (2) we 
have a € t;41. Continuing in this manner, we either get an i where a € t; 
or we get Oa € tp. Since tẹ is the root, this will give us a € tp. Q.E.D. 


Lemma A.3. For all t € AT, o ~; B ¢ t implies there exists a path 


pe : t= ti Sar te... AT tk which conforms to ø such that one of 
the following conditions holds: 


o BE tk. 
e tų is a non-leaf node and moves(t,) N o(tk) = ¥. 


Proof. Consider the least set R containing t and closed under the following 
condition: 


e if tı € R then for every transition tı — tz such that a € o(t,) we 
have t2 € R. 


If there exists an atom t/ € R such that 6 ¢ t’ or if t’ is a non-leaf node 
and moves(t’) N a(t’) = y, then we are done. Suppose not, then we have 
+ R> 8 and F (RA-leaf) > Vaen((a) True A (o): : a). 

Claim A.4. The following are derivable. 


1. F (RAT; A (0): : a) > [a] È. 


2. H (rA R) > [N]È. 


Assume claim A.4 holds, then applying (IND) rule we get + R> o~s B. 
But t € R and therefore F t > o ~; p, contradicting the assumption that 
au, BET. Q.E.D. 


Proof. To prove 1, suppose the claim does not hold. We have that (RA 7; A 
(a); : a) A (a)5R is consistent. Let R’ = AT — R. If R' = y then R= AT 
in which case its easy to see that the claim holds. If R’ 4 ọ, then we have 
(RATA Q ); : a) A (a)R’ is consistent. Hence for some tı € R and tz € R’, 
we have (fi AT; A (ø); : a)) A (a)ég is consistent. Which implies tı “4.47 te 
and this transition conforms to ø. By closure condition on R, t2 € R which 
gives us the required contradiction. 


Proof of 2 is similar. Q.E.D. (Claim A.4) 
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Abstract 


Several formal models of awareness have been introduced in both 
computer science and economics literature as a solution to the prob- 
lem of logical omniscience. In this chapter, I provide a philosophical 
discussion of awareness logic, showing that its underlying intuition 
appears already in the seminal work of Hintikka. Furthermore, I 
show that the same intuition is pivotal in Newell’s account of agency, 
and that it can be accommodated in Levi’s distinction between epis- 
temic commitment and performance. In the second part of the chap- 
ter, I propose and investigate a first-order extension of Fagin and 
Halpern’s Logic of General Awareness, tackling the problem of repre- 
senting “awareness of unawareness”. The language is interpreted over 
neighborhood structures, following the work of Arlé-Costa and Pacuit 
on First-Order Classical Modal Logic. Adapting existing techniques, 
I furthermore prove that there exist useful decidable fragments of 
quantified logic of awareness. 


1 Introduction 


Since its first formulations (cf. [21]), epistemic logic has been confronted 
with the problem of logical omniscience. Although Kripkean semantics ap- 
peared to be the natural interpretation of logics meant to represent knowl- 
edge or belief, it implies that agents are reasoners that know (or at least are 
committed to knowing) every valid formula. Furthermore, agents’ knowl- 
edge is closed under logical consequence, so that if an agent knows y and w 
is a logical consequence of y, then the agent knows ~ as well. If we focus on 
representing pure knowledge attributions, rather than attributions of epis- 
temic commitment, such a notion of knowledge (or belief) is too strong to 
be an adequate representation of human epistemic reasoning. It is possible 
to attack the problem by building into the semantics a distinction between 
implicit and explicit knowledge (or belief). The intuition behind such a 
distinction is that an agent is not always aware of all propositions. In par- 
ticular, if y is a valid formula, but the agent is not aware of it, the agent is 
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said to know y implicitly, while she fails to know it explicitly. The agent ex- 
plicitly knows y, on the other hand, when she both implicitly knows y and 
she is aware of y. In their fundamental article [9], Fagin and Halpern for- 
mally introduced the concept of awareness in the context of epistemic logic, 
providing semantic grounds for the distinction between implicit and explicit 
belief. The technical concept of “awareness” they introduce is amenable to 
different discursive interpretations that can be captured by specific axioms. 

In the last decade, recognizing the importance of modeling asymmetric 
information and unforeseen consequences, economists have turned their at- 
tention to epistemic formalizations supplemented with (un)awareness (cf. 
(32, 33]), and noticed that partitional structures as introduced by Aumann 
cannot represent awareness [8]. The model in [33] defines awareness ex- 
plicitly in terms of knowledge. An agent is said to be aware of y iff she 
knows y or she both does not know that p and knows that she does not 
know y. Halpern ([13]) shows that such a model is a particular case of 
the logic of awareness introduced in [9]. Heifetz et al. ([20]) present a set- 
theoretical model that generalizes traditional information structures à la 
Aumann. Its axiomatization in a 3-valued epistemic logic is provided by by 
Halpern and Régo in [14]. A further, purely set-theoretical model of aware- 
ness is given by Li ([30]). Awareness, or lack thereof, plays an important 
role in game-theoretic modeling. Recently, a significant amount of litera- 
ture has appeared in which the issue of awareness in games is taken into 
account. Feinberg ({11]) incorporates unawareness into games and shows 
that unawareness can lead to cooperative outcomes in the finitely repeated 
prisoner’s dilemma. A preliminary investigation on the role of awareness 
in the context of game-theoretical definitions of convention is performed by 
Sillari ([37]). Haleprn and Rêgo ([15]) define extensive-form games with 
possibly unaware players in which the usual assumption of common knowl- 
edge of the structure of the game may fail. Heifetz et al. ([19]) take into 
account Bayesian games with unawareness. In [29] the concept of subgame 
perfect equilibrium is extended to games with unawareness. 

This paper makes two main contributions to the literature on aware- 
ness. On the one hand, I provide philosophical underpinnings for the idea 
of awareness structures. On the other, I propose a new system of first-order 
epistemic logic with awareness that offers certain advantages over existing 
systems. As for the first contribution, I build on epistemological analyses of 
the problem of logical omniscience. Although the authors I consider need 
not align themselves with advocates of the awareness structures solution, 
I argue in the following that their analyses are not only compatible with 
formal models of awareness, but also compelling grounds for choosing them 
as the appropriate solution to the logical omniscience problem. I consider, 
for example, Levi’s idea of epistemic commitment. In a nutshell, ideally 
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situated agents possess, in their incorrigible core of knowledge, all logical 
truths, and the agents’ bodies of knowledge are closed under implication. 
Although agents are committed by (ideal) standards of rationality to hold- 
ing such propositions as items of knowledge, actual agents are aware only 
of a subset of them (cf. [25, pp. 9-13]). Furthermore, I consider Newell’s 
theory of agency (as advanced in [35]) and show that it contains a fore- 
shadowing of the notion that awareness allows us to discriminate between 
an agent’s explicit and implicit knowledge. Although Newell’s analysis is 
conducted at a fairly abstract level, it is arguable that he is endorsing a rep- 
resentation model in which knowledge explicitly held by a system is given 
by its (implicit) knowledge plus some kind of access function (cf. in partic- 
ular [35, p. 114]). It is not hard to see that this intuition corresponds to the 
intuition behind awareness structures. Finally, I argue that the intuition 
behind Hintikka’s own treatment of logical omniscience in [21] can also be 
considered as related to awareness structures in a precise sense that will be 
elucidated in the following. 

As for the second contribution, I identify two main motivations for the 
introduction of a new formal system of awareness logic. First and foremost, 
it addresses the problem of limited expressivity of existing (propositional) 
logics of awareness. Indeed, [16] notice that both standard epistemic logic 
augmented with awareness and the awareness models set forth in the eco- 
nomics literature cannot express the fact that an agent may (explicitly) 
know that she is unaware of some proposition without there being an ex- 
plicit proposition that she is unaware of. This limitation of the existing 
models needs to be overcome, since examples of “knowledge of unawareness” 
are often observed in actual situations. Consider Levi’s idea of commit- 
ment mentioned above: we are committed to knowing (in fact, we explicitly 
know) that there exists a prime number greater than the largest known 
prime number, although we know that we do not know what number that 
is. Or, consider David Lewis’s theory of convention! as a regularity in the 
solution of a recurrent coordination game: when trying to learn what the 
conventional behavior in a certain environment might be, an agent might 
know (or, perhaps more interestingly, deem highly probable) that there is a 
conventional regularity, without having yet figured out what such a regular- 
ity actually is. Or, in the context of a two-person game with unawareness, 
a player might explicitly know that the other player has some strategy at 
her disposal, yet not know what such a strategy might be. Halpern and 
Régo propose in [16] a sound and complete second-order propositional epis- 
temic logic for reasoning about knowledge of unawareness. However, the 
validity problem for their logic turns out to be no better than recursively 


1 Cf. [28] and the reconstruction offered in [37], in which awareness structures find a 
concrete application. 
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enumerable, even in the case of S5, which was proven to be decidable in 
[12]. Halpern and Rêgo conjecture that there are three causes for undecid- 
ability, each one sufficient: (i) the presence of the awareness operators, (ii) 
the presence of more than one modality, (iii) the absence of Euclidean rela- 
tions. Undecidability of second-order, multi-modal S5 should not come as 
a surprise. For example, [1] shows that adding a second modality to second- 
order S5 makes it equivalent to full second-order predicate logic. My aim 
is to present a decidable logic for reasoning about knowledge of unaware- 
ness. The strategy I adopt consists in extending predicate modal logic with 
awareness operators and showing that it allows to represent knowledge of 
unawareness. Using the techniques introduced in [40] and [39], I can then 
isolate useful decidable fragments of it. 

There is a further reason for the introduction of predicate epistemic 
logic with awareness. The extension from propositional to predicate logic 
takes place in the context of classical systems interpreted over neighbor- 
hood structures (cf. [6]), rather than in the traditional framework of normal 
systems interpreted over Kripke structures. In so doing, I aim at bringing 
together the recent literature (cf. [2], [3]) on first-order classical systems for 
epistemic logic and the literature on awareness structures. The rationale 
for this choice lies in the fact that I intend to formulate a system in which 
Kyburg’s ‘risky knowledge’ or Jeffrey’s ‘probable knowledge’ is expressible 
as high probability (or even as probability one belief, as Aumann does in 
the game-theoretical context). High probability operators give rise to Ky- 
burg’s lottery paradox, which, in the context of first-order epistemic logic 
(cf. [3]) can be seen as an instance of the Barcan formulas. Thus, first-order 
Kripke structures with constant domains, in which the Barcan formula is 
validated, cease to be adequate models. The use of neighborhood structures 
allows us to work with constant domains without committing to the validity 
of the Barcan formulas (cf. [2]), hence presents itself as a natural candidate 
for modeling high probability operators. The second-order logic of Halpern 
and Rêgo also requires the Barcan formulas to be validated, and hence does 
not lend itself to the modeling of knowledge as high-probability operators. 

The rest of the paper is organized as follows: In the Section 2, I review 
and discuss the philosophical accounts of logical omniscience offered by Hin- 
tikka, Newell and Levi, stress their structural similarities, and show how 
these accounts compare with the intuition underlying Fagin and Halpern’s 
logic of awareness. In Section 3, I build on Arló-Costa and Pacuit’s version 
of first-order classical systems of epistemic logic, augmenting them with 
awareness structures. I then show that such a quantified logic of aware- 
ness is expressive enough to represent knowledge of unawareness and that 
Wolter and Zakharyashev’s proof of the decidability of various fragments 
of first-order multi-modal logic (cf. [40]) can be modified to carry over to 
quantified logic of awareness. 
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2 Logical Omniscience 


In this section, I consider accounts of the problem of logical omniscience 
provided in Hintikka’s presentation of epistemic logic, Newell’s theory of 
agency and Levi’s epistemology. I show through my analysis that all such 
approaches to logical omniscience share a common structure, and that Fagin 
and Halpern’s logic of awareness has reference to such a structure. 


2.1 Hintikka: Information and justification 


Hintikka’s essay Knowledge and Belief is commonly regarded as the seminal 
contribution to the development of epistemic logic. Logical omniscience is 
an essential philosophical element in Hintikka’s conceptual analysis, as well 
as in the formal construction stemming from it. Consider, for instance, the 
following quote: 


It is true, in some sense, that if I utter (10) ‘I don’t know whether 
p’ then I am not altogether consistent unless it really is possible, 
for all that I know, that p fails to be the case. But this notion of 
consistency is a rather unusual one, for it makes it inconsistent for 
me to say (10) whenever p is a logical consequence of what I know. 
Now if this consequence-relation is a distant one, I may fail to know, 
in a perfectly good sense, that p is the case, for I may fail to see that 
p follows from what I know”. 


Hintikka notices in [21, p. 23] that we need to distinguish two senses of 
“knowing”. A first, weak, kind of knowledge (or belief) is simply concerned 
with the truth of a proposition p. In natural language, this is the sense 
of “knowing p” related to “being conscious? that p”, or “being informed 
that p” or “being under the impression that p”, etc. The second, stronger, 
sense of knowing is not only concerned with the truth of p, but also with the 
justification of the agent’s knowledge. According to different epistemological 
accounts, “knowing” in this latter sense may mean that the agent has “all 
the evidence needed to assert p”, or has “the right to be sure that p”, or has 
“adequate evidence for p”, etc. Whichever of these epistemological stances 
one chooses, the strong sense of knowing incorporates both the element 
of bare “availability” of the truth of p (information) and the element of 
the epistemological justification for p. Such a distinction is essential in 
Hintikka’s analysis of the notion of consistency relative to knowledge and 
belief, which in turn is crucial for the design of his formal system. 


2 Cf. [21], or p. 25 of the 2005 edition of the book, from which the page references are 
drawn hereafter. 

3 Referring to the weak sense of “knowing”, Hintikka mentions a natural language ex- 
pression as “the agent is aware of p”. In order to avoid confusion with the different, 
technical use of “awareness”, in this context I avoid the term “awareness” altogether. 
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Syntactically, Hintikka’s system does not essentially differ from the epis- 
temic systems that have come to prevail in the literature, the only notable 
difference being the explicit mention of the “dual” of the knowledge opera- 
tor, P;, to be read as “it is compatible with all 7 knows that...”. The pursuit 
of consistency criteria for the notions of knowledge and belief moves from 
the analysis of sets of formulas in which both knowledge and “possibility” 
operators are present. The main idea is that if the set {Kipi,..., Kipn, Pig} 
is consistent, then the set {Kip1,..., Kipn,q} is also consistent. The dis- 
tinction between the two senses of “knowing” above is crucial to the justi- 
fication of this idea. If “knowing p” is taken in the weak sense of “being 
conscious of p”, then a weaker notion of consistency is appropriate, accord- 
ing to which if {Kjpi,..., Kipn, Piq} is consistent, then {pi,...,pn,q} is 
consistent as well. Such a weaker notion, however, is no longer sufficient 
once we interpret K;p as “i is justified in knowing p”, according to the 
stronger sense of knowing. In this case, q has to be compatible not just 
with the truth of all statements p1,..., Pn, but also with the fact that i is 
in the position to justify (strongly know) each of the pi,...,pn, that is to 
say, q has to be consistent with each one of the K;p1,..., Kipn. 

Other criteria of consistency are those relative to the knowledge operator 
(if A is a consistent set and contains K;p, then AU p is consistent), to the 
boolean connectives (for instance, if À is consistent and contains p ^q, then 
AU {p,q} is consistent), and to the duality conditions (if A is consistent 
and —~K;p € A, then À U P;-7p is consistent; while if ~P;p € A, then AU 
K;7p is consistent). The duality conditions trigger the problem of logical 
omniscience. Consider again the quote at the onset of this subsection: if 
Kiq holds, and p is a logical consequence of q, then —K;p is inconsistent. 
Thus, at this juncture, a modeling decision has to be made. If we want 
to admit those cases in which an agent fails to know a logical consequence 
of what she knows, either (i) we may tweak the notion of knowledge in 
a way that makes such a predicament consistent, or (ii) we may dispense 
with the notion of consistency, weakening it in a way that makes such a 
predicament admissible. The two routes, of course, lead to different formal 
models. Hintikka chooses the latter strategy, while epistemic systems with 
awareness à la Fagin and Halpern choose the former. However, the two 
routes are two faces of the same coin. Hintikka’s concept of defensibility, 
intended as “immunity from certain standards of criticism” ([21, p. 27]), 
replacing the notion of consistency, allows us to consider knowledge (of 
the kind that allows for logical omniscience to fail) as the intersection of 
both the weak and the strong sense of “knowing” above, in a way that, 
at least structurally, is not far from considering explicit knowledge as the 
intersection of implicit knowledge and awareness in [9]. 
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To make more precise the notion of defensibility as “immunity from cer- 
tain standards of criticism”, and to see more clearly the similarity with 
awareness logic, let me briefly summarize Hintikka’s formal system. Hin- 
tikka’s semantics is kindred in spirit to possible worlds structures. There 
are, however, proceeding from the notion of defensibility, important differ- 
ences with standard possible worlds semantics. First, define a model set, 
with respect to boolean connectives, as a set of formulas such that 


peppy ( 
(p\qg)€ bape pandgepy (A 
( 


) 

) 

(pVq)eu>peporgeu v) 
~pEup>pEp (=>) 
(pAg) € “> 7p € wor 7q € ps (A) 
~(p Vq) > 7p € u and =q € p (v) 


In order to add epistemic operators to model sets, Hintikka postulates 
the existence of a set of model sets (called the model system Q) and of an 
alternativeness relation for each agent, and adds the clauses 


If Pip € p, then there exists at least a u“ such that u* 

is an alternative to p for i, and p € p* 

If Kip € p, then, if u“ is an alternative to u for i, then Kip € u* (KK) 
If Kip € p, then p € u (K) 


Thus, we have consistent sets of formulas constituting a model system, 
an accessibility relation between model sets in the system for each agent, 
and a semantic account of knowledge close to the standard Kripkean one 
(to see that, notice that KK and K taken together imply that if Kip € u 
then p € u* for all u* alternative to u in Q). The fundamental difference 
with Kripke models lies in the elements of the domain: model sets (i.e., 
consistent sets of formulas) in Hintikka’s semantics, possible worlds (i.e., 
maximally consistent sets of formulas) in Kripke’s. Thus, Hintikka’s model 
sets are partial descriptions of possible worlds.* 


4 Hintikka has made this claim unexceptionable in later writings: “The only viable 
interpretation of logicians’ “possible worlds” is the one that I initially assumed was 
intended by everyone. That is to understand “possible worlds” as scenarios, that is, 
applications of our logic, language or some other theory to some part of the universe 
that can be actually or at least conceptually isolated sufficiently from the rest”, cf. [23, 
p. 22]. But cf. also [21, pp. 33-34]: “For our present purposes, the gist of their [model 
sets] formal properties may be expressed in an intuitive form by saying that they 
constitute [...] a very good formal counterpart to the informal idea of a (partial) 
description of a possible state of affairs” (emphasis added). 
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The notion of defensibility is now definable as follows: a set of formulas 
is defensible iff it can be embedded in a model set of a model system. As 
the notion of consistency is replaced with that of defensibility, the notion of 
validity is replaced with that of self-sustenance. It follows easily from the 
definitions that p — q is self-sustaining iff the set p, ~q is not defensible’. 
This is key for overcoming logical omniscience: although an agent knows q 
if she knows p and p — q is self-sustaining, it need not be the case that she 
knows q if she knows p and p — q is valid, since, in this case, p — q need not 
be self-sustaining®. This may occur if q does not appear in the model sets 
of Q, so that p,q is embeddable in them, making ~K;q defensible (since, 
by the duality rule, P;-q € u and, by rule [K], there exists a u* such that 
aq € p*). Thus, =~K;q is defensible as long as q can be kept out of some 
model set u, provided that 2 does not incur in criticism according to certain 
epistemic standards. That is, for an agent to be required to know q it is 
not enough, say, that q logically follows from the agent’s knowledge, but it 
also needs to be the case that q belongs to a model set u. Similarly’, in [9], 
for an agent to know y explicitly, it is not sufficient that y logically follows 
from the agent’s knowledge, but it also needs to be the case that y belongs 
to the agent’s awareness set. In this sense, a formula not appearing in a 
model set and a formula not belonging to an awareness set may be regarded 
as cognate notions. 


2.2 Newell: Knowledge and access 


The interest of the AI community in the logic of knowledge and its rep- 
resentation does not need to be stressed here. Intelligent agents must be 
endowed with the capability of reasoning about the current state of the 
world, about what other agents believe the current state of the world is, 
etc. Planning, language processing, distributed architectures are only some 
of the many fields of computer science in which reasoning about knowledge 
plays a central role. It is not surprising, then, that computer scientists paid 
attention to the epistemic interpretation of modal logics and, hence, that 
they had to confront the problem of logical omniscience. It is difficult (and 
probably not particularly relevant) to adjudicate issues of precedence, but 


5 If the set {p, ~q} is not defensible, then it cannot be embedded in any model set u, 
meaning that either ~p or q (or both) must belong to u, making p —> q self-sustaining, 
and vice versa. 

Notice however [21, p. 46] that other aspects of logical omniscience are present: the 
valid formula (Kip ^ Kiq) > Ki(p A q) is also self-sustaining. 

A formal proof is beyond the scope of this paper, and the interested reader can find it 
in [38]. To see the gist of the argument, consider that model sets are partial description 
of possible worlds. While one can (as it is the case, e.g., in [24]) model the distinction 
between explicit and implicit knowledge by resorting to partial descriptions of possible 
worlds, one can, equivalently, do so by “sieving” the description of a possible world 
through awareness sets. 


a 
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the idea of using some conceptualization of awareness to cope with the prob- 
lem of logical omniscience appeared in the early 80s, possibly on a cue by 
Alan Newell. In 1980, Newell delivered the first presidential address of the 
American Association for Artificial Intelligence. The title was The knowl- 
edge level, and the presidential address is reproduced in [35]. The article 
focuses on the distinction between knowledge and its representation, both 
understood as functional components of an intelligent system. 

An intelligent system, in the functional view of agency endorsed by 
Newell, is embedded in an action-oriented environment. The system’s ac- 
tivity consists in the process from a perceptual component (that inputs 
task statements and information), through a representation module (that 
represents tasks and information as data structures), to a goal structure 
(the solution to the given task statement). In this picture, knowledge is 
perceived from the external world, and stored as it is represented in data 
structures. Newell claims that there is a distinction between knowledge and 
its representation, much like there is a one between the symbolic level of a 
computer system and the level of the actual physical processes supporting 
the symbolic manipulations. A level in a computer system consists of a 
medium (which is to be processed), components together with laws of com- 
position, a system, and laws that determine the behavior of the system. For 
example, at the symbolic level the system is the computer, its components 
are symbols and their syntax, the medium consists of memories, while the 
laws of behavior are given by the interpretation of logical operations. Be- 
low the symbolic level, there is the physical level of circuits and devices. 
Among the properties of levels, we notice that each level is reducible to the 
next lower level (e.g., logical operations in terms of switches), but also that 
a level need not have a description at higher levels. Newell takes it that 
knowledge constitutes a computer system level located immediately above 
the symbolic level. 

At the knowledge level, the system is the agent; the components are 
goals, actions and bodies (of knowledge); the medium is knowledge and the 
behavioral rule is rationality. Notice that the symbolic level constitutes the 
level of representation. Hence, since every level is reducible to the next lower 
level, knowledge can be represented through symbolic systems. But can we 
provide a description of the knowledge level without resorting to the level 
of representation? It turns out that we can, although we can only insofar 
as we do not decouple knowledge and action. In particular, says Newell, “it 
is unclear in what sense [systems lacking rationality] can be said to have 
knowledge®”, where “rationality” stands for “principles of action”. Indeed 
an agent, at the knowledge level, is but a set of actions, bodies of knowl- 
edge and a set of goals, rather independently of whether the agent has any 


8 Cf. [35, p. 100]. 
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physical implementation. What, then, is knowledge? Knowledge, according 
to Newell, is whatever can be ascribed to an agent, such that the observed 
behavior of the agent can be explained (that is, computed) according to the 
laws of behavior encoded in the principle of rationality. The principle of 
rationality appears to be unqualified: “If an agent has knowledge that one 
of its actions will lead to one of his goals, then the agent will select that 
action®”. Thus, the definition of knowledge is a procedural one: an observer 
notices the action undertaken by the agent; given that the observer is famil- 
iar with the agent’s goals and its rationality, the observer can therefore infer 
what knowledge the agent must possess. Knowledge is not defined struc- 
turally, for example as physical objects, symbols standing for them and their 
specific properties and relations. Knowledge is rather defined functionally 
as what mediates the behavior of the agent and the principle of rationality 
governing the agent’s actions. Can we not sever the bond between knowl- 
edge and action by providing, for example, a characterization of knowledge 
in terms of a physical structure corresponding to it? As Newell explains, 
“the answer in a nutshell is that knowledge of the world cannot be captured 
in a finite structure. [...] Knowledge as a structure must contain at least as 
much variety as the set of all truths (i.e., propositions) that the agent can 
respond tot”. Hence, knowledge cannot be captured in a finite physical 
structure, and can only be considered in its functional relation with action. 

Thus (a version of) the problem of logical omniscience presents itself 
when it comes to describing the epistemic aspect of an intelligent system. 
Ideally (at the knowledge level), the body of knowledge an agent is equipped 
with is unbounded, hence knowledge cannot be represented in a physical 
system. However, recall from above how a level of interpretation of the 
intelligent system is reducible to the next lower level. Knowledge should 
therefore be reducible to the level of symbols. This implies that the sym- 
bolic level necessarily encompasses only a portion of the unbounded body of 
knowledge that the agent possesses. It should begin to be apparent, at this 
point, that what Newell calls “knowledge” is akin to what in awareness epis- 
temic logic is called “implicit knowledge,” whereas what Newell refers to as 
“representation” corresponds to what in awareness logic is called “explicit 
knowledge”. Newell’s analysis endorses the view that explicit knowledge 
corresponds to implicit knowledge and awareness as witnessed by the “slo- 
gan equation!!” 


Representation = Knowledge + Access. 


The interesting question is then: in which way does an agent extract rep- 


9 CE. [35, p. 102]. Although Newell, in the following, refines it, his principle of rationality 
does not seem to be explicitly concerned with utility. 

10 Cf. [35, p. 107]. 

11 Cf. [35, p. 114]. 
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resentation from knowledge? Or, in other terms: Given the definition of 
representation above, what can its theory be? Building a theory of rep- 
resentation involves building a theory of access (that is, of awareness), to 
explain how agents manage to extract limited, explicit knowledge (working 
knowledge, representation) from their unbounded implicit knowledge. The 
suggestive idea is that agents do so “intelligently”, i.e., by judging what is 
relevant to the task at hand. Such a judgment, in turn, depends on the 
principle of rationality. Hence, knowledge and action cannot be decoupled 
and knowledge cannot be entirely represented at the symbolic level, since 
it involves both structures and processes!*. Given the “slogan equation” 
above, it seems that one could identify such processes with explicit and ef- 
fective rules governing the role of awareness. Logics, as they are “one class 
of representations |...] uniquely fitted to the analysis of knowledge and 
representation!®”, seem to be suitable for such an endeavor. In particular, 
epistemic logics enriched with awareness operators are natural candidates 
to axiomatize theories of explicit knowledge representation. 


2.3 Levi: Commitment and performance 


Levi illustrates (in [25]) the concept of epistemic commitment through the 
following example: an agent is considering what integer stands in the bil- 
lionth decimal place in the decimal expansion of m. She is considering ten 
hypotheses of the form “the integer in the billionth decimal place in the dec- 
imal expansion of m is j”, where j designates one of the first ten integers. 
Exactly one of those hypotheses is consistent with the logical and mathe- 
matical truths that, according to Levi, are part of the incorrigible core of 
the agent’s body of knowledge. However, it is reasonable to think that, if 
the agent has not performed (or has no way to perform) the needed calcu- 
lations!*, “there is an important sense in which [the agent] does not know 
which of these hypotheses is entailed by [logical and mathematical truths 
and, hence, does not know what the integer in the billionth place in the 
decimal expansion of 7 ist”. Levi stresses that the agent is committed to 
believing the right hypothesis, but she may at the same time be unaware of 
what the right hypothesis is. While the body of knowledge of an ideally sit- 


12 The idea is taken up again in [5], where a broader, partly taxonomical analysis of 
(artificial) agency is carried out. Moving along a discrete series of models of agents 
increasingly limited in their processing capabilities, we find, at the “fully idealized” 
end of the spectrum, the omnipotent, logically omniscient agent. Next to it, we find 
the rational agent, which, as described above, uses the principle of rationality to sieve 
its knowledge and to obtain a working approximation of it. 

13 Cf. [35, p. 100] 

14 It should be clear to the reader that Levi’s argument carries over also to those cases 
in which the lack of (explicit) knowledge follows from reasons other than lack of com- 
putational resources. 

15 Cf. [25, pp. 9-10]. 
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uated and rational agent contains all logical truths and their consequences, 
the body of knowledge of real persons or institutions does not. Epistemic 
(or, as Levi prefers, doxastic) commitments are necessary constituents of 
knowledge, which, although ideally sufficient to achieve knowledge, must in 
practice be supplemented with a further element. As Levi puts it: “I do 
assume, however, that to be aware of one’s commitment is to know what 
they are” 16, 

The normative aspect of the principle of rationality regulating epistemic 
commitments and, hence, their relative performances, is further explored 
in [27]. Levi maintains that the principle of rationality in inquiry and de- 
liberation is twofold. On the one hand, it imposes necessary, but weak, 
coherence conditions on the agent’s state of full belief, credal probability, 
and preferences. On the other, it provides minimal conditions for the jus- 
tification of changes in the agent’s state of full belief, credal probability, 
and preferences. As weak as the coherence constraints might be, they are 
demanding well beyond the capability of any actual agent. For instance, 
full beliefs should be closed under logical consequence; credal probabilities 
should respect the laws of probability; and preferences should be transitive 
and satisfy independence conditions. Hence, such principles of rationality 
are not to be thought of as descriptive (or predictive) or, for that matter, 
normative (since it is not sensible to impose conditions that cannot possi- 
bly be fulfilled). They are, says Levi, prescriptive, in the sense that they do 
not require compliance tout court, but rather demand that we enhance our 
ability to follow them. 

Agents fail to comply with the principle of rationality requiring the de- 
ductive closure of their belief set, and they do so for multiple reasons. An 
agent might fail to entertain a belief logically implied by other beliefs of hers 
because she is lacking in attention. Or, being ignorant of the relevant de- 
ductive rules, she may draw an incorrect conclusion or even refuse to draw a 
conclusion altogether. The former case, according to Levi, can be accommo- 
dated by understanding belief as a disposition to assent upon interrogation. 
In the latter, the agent needs to improve her logical abilities—by “seeking 
therapy”. In both cases, however, what is observed is a discrepancy between 
the agent’s commitment to hold an epistemic disposition, and her epistemic 
performance, which fails to display the disposition she is committed to hav- 
ing. The prescriptive character of the principle of rationality gives the agent 
an (epistemic) obligation to fulfill the commitment to full belief. The agent 
is thus committed!’ to holding such a belief. The notion of full belief ap- 


16 Cf. [25, p. 12]. 

17 She is committed in the sense (see [27]) in which one is committed to keep a religious 
vow to sanctity: occasional sinning is tolerated, and the vow is to be considered upheld 
as long as the pious agent strives to fulfill the commitments the vow implies. However, 
going back to the principle of rationality, one should notice that “epistemic therapy” 
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pears both as an epistemic disposition (commitment) of the agent, as well 
as the actual performance of her disposition. 

The discussion of Levi's idea of epistemic commitment provides us with 
three related, yet distinct concepts involved in the description of the epis- 
temic state of an agent. On the one hand, we have epistemic commitments 
(which we could think of as implicit beliefs). On the other, we have com- 
mitments that the agent fulfills, that is to say, in the terminology of [25], 
commitments of which the agent is aware (we could think of those as explicit 
beliefs). The latter, though, calls for a third element, the agent’s awareness 
of the commitment she is going to fulfill. 


2.4 Logical omniscience and awareness logic 


The three examinations of the problem of logical omniscience described here 
do not deal directly with the logic of awareness, and actually all of them 
pre-date even the earliest systems of awareness logic (for instance, [24]). In 
fact, Hintikka’s position on the issue has shifted over the years. Since the 
introduction of Rantala’s “urn models” (cf. [36]), the author of Knowledge 
and Belief has endorsed the “impossible worlds” solution to the problem 
of logical omniscience (cf. [22]). In the case of Isaac Levi’s approach, it is 
at best doubtful that Fagin and Halpern understand the notions of implicit 
and explicit knowledge in the same way Levi elaborates those of epistemic 
commitment and epistemic performance. However, the three accounts ana- 
lyzed above do share a common structure, whose form is captured by Fagin 
and Halpern’s logic of awareness. In the case of Allen Newell’s analysis 
of agency at the knowledge level, there is a marked conceptual proximity 
between Newell’s notions of knowledge, representation and access, on the 
one hand, and Fagin and Halpern’s notions of implicit knowledge, explicit 
knowledge and awareness, on the other. But consider also Hintikka’s dis- 
tinction between a weak and a strong sense of knowing, the former roughly 
related to the meaning of “having information that”, the latter to the one of 
“being justified in having information that”. If we interpret “awareness” as 
meaning “having a justification”, then strong knowledge is yielded by weak 
knowledge and justification, just as explicit knowledge is yielded by im- 
plicit knowledge and awareness. Also Levi’s distinction between epistemic 
commitment and epistemic performance can be operationalized by stipu- 
lating that epistemic performance stems from the simultaneous presence of 
both the agent’s epistemic commitment and the agent’s recognition of her 
own commitment, just as explicit knowledge is yielded by the simultaneous 


comes at a cost (of time, resources, effort etc.) and that, moreover, not all our doxastic 
commitments (actually only a few of them) are epistemically useful (think of the belief 
that p, which implies the belief that p V p, that p V p V p, and so on). Hence the idea 
of “seeking therapy” or of using “prosthetic devices” to comply with the principle of 
rationality leaves space for further qualification. 
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presence of implicit knowledge and awareness. 

Fagin and Halpern’s logic of awareness was meant to be a versatile for- 
mal tool in the first place!8, in such a way that its purely formal account 
of “awareness” could be substantiated with a particular (concrete) inter- 
pretation of “awareness”. Such an interpretation could be epistemological 
justification, as in the case of Hintikka’s account; or it could be physical 
access, as in case of Newell’s artificial agent; or it could be psychological 
awareness, as in the case of Levi’s flesh-and-blood agents. All three interpre- 
tations fit the general structure of Fagin and Halpern’s awareness logic. It is, 
however, much less clear whether it is possible to capture axiomatically the 
different properties that “awareness” enjoys in the philosophical accounts 
delineated in the previous subsections. From a normative standpoint, one 
would need to answer the question: given that the agents capabilities are 
bounded, which are the items of knowledge that (bounded) rationality re- 
quires that the agent explicitly hold? This line of inquiry is pursued by 
Harman (cf. [18]) and by Cherniak (cf. [7]). Levi notices that the “therapy” 
to be undertaken in order to better our epistemic performance comes at a 
cost, triggering a difficult prescriptive question as to how and how much an 
agent should invest to try and better approximate her epistemic commit- 
ment. Levi does not seem to think that such a question can be answered 
in full generality’. It seems to me that there is an important dynamic 
component to the question (if one’s goal is such-and-such, then she should 
perform epistemically up to such-and-such portion of her epistemic commit- 
ment) that is well captured in Newell’s intuition that knowledge representa- 
tion and action cannot be decoupled in a physical system. The formidable 
task of providing an axiomatic answer to the normative question about the 
relation between implicit and explicit knowledge lies beyond the scope of 
this contribution, and in the formal system advanced in the next section, I 
will consider only those properties of awareness that are now standard in 
the literature. 

18 Cf, [9, p. 41]: “Different notions of knowledge and belief will be appropriate for different 
applications. We believe that one of the contributions of this paper is providing tools 
for constructing reasonable semantic models of notions of knowledge with a variety of 
properties.” Also, “once we have a concrete interpretation [of awareness] in mind, we 
may well add some restrictions to the awareness functions to capture certain properties 
of ‘awareness’,” ibidem, p. 54. 

“A lazy inquirer may regard the effort to fulfill his commitments as too costly where 
a more energetic inquirer suffering from the same disabilities does not. Is the lazy 
inquirer failing to do what he ought to do to fulfill his commitments, in contrast to 
the more energetic inquirer? I have no firm answer to this question [...] We can 


recognize the question as a prescriptive one without pretending that we are always in 
the position to answer it in advance.”, [26, p. 168, n. 14]. 
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3 First-Order Logic of Awareness 


In this section, I extend Fagin and Halpern’s logic of general awareness 
(cf. [9]) to a first-order logic of awareness, show that awareness of unaware- 
ness can be expressed in the system, and prove that there exist decidable 
fragments of the system. For a general introduction to propositional epis- 
temic logic, cf. [31] and [10]. For detailed treatments of the first-order epis- 
temic systems, cf. [6] and the work of Arlé6-Costa and Pacuit ([2] and [3]). 


3.1 First-order classical models 


The language Ln of multi-agent first-order epistemic logic consists of the 
connectives ^A and ~, the quantifier Y, parentheses and n modal operators 
Kı,..., Kn, one for each agent considered in the system. Furthermore, we 
need a countable collection of individual variables V and a countable set 
of n-place predicate symbols for each n > 1. The expression y(x) denotes 
that x occurs free in y, while y[x/y] stands for the formula y in which the 
free variable x is replaced with the free variable y. An atomic formula has 
the form P(z1,..., £n), where y is a predicate symbol of arity n. If S isa 
classical propositional modal logic, QS is given by the following axioms: 


All axioms from S (S) 
vzo(z) > ply/2] (v) 
From y —> y infer y — Yxy, where z is not free in y. (Gen) 


In particular, if S contains the only modal axiom E (from y © y, infer 
Kip => Kip) we have the weakest classical system E; if S validates also 
axiom M (Ki(pA wv) > (Kip A Kiw)), we have system (E)M, etc. (see [6] 
for an exhaustive treatment of classical systems of modal logic). 

As to the semantics, a constant domain neighborhood frame is a tuple 
F =(W,M,...,Nn,D), where W is a set of possible worlds, D is a non- 
empty set called the domain, and each M; is a neighborhood function from 
W to 2?" . If we define the intension (or truth set) of a formula ọ to be 
the set of all worlds in which y is true, then we can say, intuitively, that 
an agent at a possible world knows all formulas whose intension belongs to 
the neighborhood of that world. A model based of a frame F is a tuple 
(W,M,..., Nn, D, I), where T is a classical first-order interpretation func- 
tion. A substitution is a function o : V > D. If a substitution o’ agrees 
with o on every variable except x, it is called an x-variant of g, and such 
a fact is denoted by the expression o ~y o’. The satisfiability relation is 
defined at each state relative to a substitution o: 


(M,w) Ko P(a1,.--,%n) iff (o(a1),..-,0(a@n)) E€ I(P,w) for each n- 
ary predicate symbol P. 
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Fo ~y iff (M, w) Eo p 
=o pAw iff (M, w) =o ẹ and (M, w) Ho w 
Fo Kig iff {v : (M, v) Ho p} € Ni(w) 


=o Vay(x) iff for each o' ~g o, (M, w) =o y(x) 


As usual, we say that a formula y is valid in M if (M,w) H ¢ for all 
worlds w in the model, while we say that ¢ is satisfiable in M if (M, w) = y 
for some worlds w in the model. Notice that QE axiomatizes first-order min- 
imal? models (in which no restrictions are placed on the neighborhoods); 
QEM axiomatizes first-order monotonic models (in which neighborhoods 
are closed under supersets); etc. 


3.2 Adding awareness 


Following [9], awareness is introduced on the syntactic level by adding to the 
language further modal operators A; and X; (with i = 1,...,n), standing 
for awareness and explicit knowledge, respectively?!. The operator X; can 
be defined in terms of K; and A;, according to the intuition that explicit 
knowledge stems from the simultaneous presence of both implicit knowledge 
and awareness, by the axiom 


Xip > Kip ^ Aig. (A0) 


Semantically, we define n functions A; from W to the set of all formu- 
las. Their values specify, for each agent and each possible world, the set 
of formulas of which the agent is aware at that particular world. Hence 
the straightforward semantic clauses for the awareness and explicit belief 
operators: 


(M, w) Fo Aig iff y € Ai(w) 


(M, w) Ko Xiy iff (M, w) = Aig and (M, w) = Kig 


In propositional awareness systems of the kind introduced in [9], different 
interpretations of awareness are captured by imposing restrictions on the 
construction of the awareness sets. For example, one can require that if 
the agent is aware of y A y, then she is aware of w and y as well. Or, one 
could require that the agent’s awareness be closed under subformulas, etc. 


20 For the terminology, see [6]. 

21 The use of neighborhood structures eliminates, of course, many aspects of the agents’ 
logical omniscience. However, axiom E is valid in all neighborhood structures. Thus, 
the distinction between implicit and explicit knowledge remains relevant, since agents 
may fail to recognize the logical equivalence of formulas y and w and, say, explicitly 
know the former without explicitly knowing the latter. 
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One of those interpretations (which, mutatis mutandis, is favored in the 
economics literature when taken together with the assumption that agents 
know what they are aware of) is that awareness is generated by primitive 
propositions. In this case, there is a set of primitive propositions ® of which 
agent 7 is aware at w, and the awareness set of i at w contains exactly 
those formulas that mention only atoms belonging to ®. Similarly, we can 
interpret awareness in a first-order system as being generated by atomic 
formulas, in the sense that i is aware of y at w iff i is aware of all atomic 
subformulas in y. Thus, for each i and w, there is a set (call it atomic 
awareness set and denote it ®;(w)) such that y € A;(w) iff p mentions 
only atoms appearing in ®;(w). Such an interpretation of awareness can 
be captured axiomatically. The axioms relative to the boolean and modal 
connectives are the usual ones (cf., e.g., [9]): 


Aimy > Aip (A1) 
Ailp Aw) > Aip A Aip (A2) 
Aikjp = Aip (A3) 
AiAjp > Aip (A4) 

Ai X;jp > Aig. (A5) 


Before discussing the axioms relative to the quantifiers, it is worth stress- 
ing that the first-order setup allows the modeler to specify some details 
about the construction of atomic awareness sets. In the propositional case, 
the generating set of primitive propositions is a list of atoms, necessarily 
unstructured. In the predicate case, we can have the atomic awareness set 
built in the semantic structure. Notice that there can be two sources of 
unawareness. An agent could be unaware of certain individuals in the do- 
main or she could be unaware of certain predicates. Consider the following 
examples: in a game of chess, (i) a player could move her knight to reach 
a position x in which the opponent’s king is checkmated; however, she can- 
not “see” the knight move and is, as a result, unaware that x is a mating 
position; or (ii) a player could move her knight, resulting in a position x 
in which the opponent’s queen is pinned; however, she is a beginner and 
is not familiar with the notion of “pinning”; she is thus unaware that x is 
a pinning position. Hence, in order for an agent to be aware of an atomic 
formula she must be aware of the individuals occurring in the interpretation 
of the formula as well as of the predicate occurring in it. It is possible to 
capture these intuitions formally: for each ¿ and w, define a “subjective 
domain” D;(w) C D and a “subjective interpretation” I; that agrees with 
I except that for some w and P of arity n, I(P, w) 4 L (P, w) = Ø. We can 
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then define the atomic awareness set for i at w by stipulating that 


(i) olzk) E€ Dilw), Yzkg,k=1,...,n 


Eiin E Oan e (o(x1),...,o(£n)) € (P, w) 


Notice that this is consistent with the notion that one should interpret 
a formula like A;y(x), where x is free in y, as saying that, under a valua- 
tion o, the agent is aware of y(x). Similarly, the truth of K;p(x) depends 
on the individual assigned to x by the valuation ø. Finally, we need to 
introduce a family of special n-ary predicates?? A!; whose intuitive meaning 
is “i is aware of objects o(x1),...,o(£n)”. Semantically, we impose that 
(M,w) =o Ali (x) iff o(x) € Di(x). 


3.3 Expressivity 


Let us now turn our attention to the issue of representing knowledge of 
unawareness. Consider the de re/de dicto distinction, and the following two 
formulas: 


The former says that agent i (explicitly) knows that there exists an 
individual enjoying property P, without her being aware of which particular 
individual enjoys P. The formula, intuitively, should be satisfiable, since 
P(x) ¢ A;(w) need not entail 3xP(x) g A;(w). On the other hand, the 
latter says that 2 is aware, of a specific x, that x has property P. If this 
is the case, it is unreasonable to admit that i can be unaware of P(x). 
By adopting appropriate restrictions on the construction of the awareness 
sets, we can design a system in which formulas like (i) are satisfiable, while 
formulas like (ii) are not. 

In particular, we need to weaken the condition that awareness is gen- 
erated by atomic formulas, since we want to allow for the case in which 
P(x) ¢ Aj(w), yet 3xP(x) € A;(w). I argue that such an interpretation of 
awareness is sensible. In fact, we may interpret P(x) not belonging to 7’s 
awareness set as meaning that 7 is not aware of a specific instance of x that 
enjoys property P, while we may interpret 4zP(x) belonging to it’s aware- 
ness set as meaning that 7 is aware that at least one specific instance of x 
(which one she ignores) enjoys property P. The versatility of the awareness 
approach is again helpful, since the blend of syntax and semantics charac- 
terizing the concept of awareness makes such an interpretation possible. 


22 Such predicates are akin to the existence predicate in free logic. However, the awareness 
system considered here is not based on free logic: the special awareness predicates will 
only be used to limit the range of possible substitutions for universal quantifiers within 
the scope of awareness operators. The behavior of quantifiers is otherwise standard. 
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Let us see in more detail what restrictions on the construction of the 
awareness sets correspond to the interpretation above. In particular, we 
want 


(i) X;d2A; P(x) to be satisfiable, while 


(ii) SvX;7A;P(x) should not be satisfiable. 


Semantically, thus, if P(x) ¢ A;(w), then (against (ii)), —A;P(x) ¢ 
Ai(w). Yet the possibility that (i) dr4A;P(x) € A;(w) is left open. The 
following condition, along with the usual conditions for awareness being 
generated by atomic formulas in the case of quantifier-free formulas, does 
the job?’ (weak 4-closure): 


If y[a/y] € Ai(w), then dry(x) € Ai(w). (*) 


It is easy to see that, if P(x) ¢ A,(w), then (ii) is not satisfiable, 
since there should exist an interpretation o’ ~, o such that (M,w) Ho 
A;7A;P(«). But that is impossible, since, for quantifier-free propositions, 
awareness is generated by atomic formulas. On the other hand, (i) entails 
that Sr4A;P(x) € A;(w), which remains satisfiable, since the weak condi- 
tion (*) does not require that —A;P(x) € A;(w). 

Let me illustrate the reason why we are considering a weak closure of 
awareness under existential quantification by means of an example: in the 
current position of a chess game, White knows (or: deems highly prob- 
able) that sacrificing the bishop triggers a mating combination, although 
she cannot see what the combination itself precisely is. Take the variable 
x to range over a domain of possible continuations of the game, and the 
predicate P to be interpreted as “is a mating combination”. Thus, at w, 
White is aware that there exists an x such that P(x). However she is not 
aware of what individual o(«) actually is (,A;P(a)), hence Aj;dx—5A;P(z) 
holds. Now, had (*) been a biconditional, since Jr—A; P(x) € A;(w) holds, 
it would have been the case that ~A;P[a/y] E€ A;(w), that is A;7A;P(y). 
In the example, White would have been aware that she is not aware that the 
specific combination o(y) led to checkmate, which is counterintuitive. The 
fact that, limited to sentences, awareness is generated by atomic formulas 
and that awareness is only weakly closed under existential quantification 
rules out such undesirable cases. 

Notice that the weak 4-closure (*) can be expressed axiomatically as 
follows: 


Aiplz/y] > Ary(2). (A6) 
What is the interplay between universal quantification and awareness? 
Consider, in the example above, the situation in which White is aware that 


23 Cf. [16], in which a similar requirement of weak existential closure is used. 
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any continuation leads to checkmate. It is then reasonable to conclude that 
she is aware, of any specific continuation she might have in mind, that it 
leads to checkmate. Thus, if, for any x, P(x) € A;(w), then P[z/y] € Ailw) 
for all y such that o(y) € D;(w). Hence the axiom 


ANap(x) > (Ali(y) > Aivl2/y)). (A7) 


This concludes the presentation of the syntax and the semantics of the 
first-order system of awareness. Various first-order modal logics are proven 
to be complete with respect to neighborhood structures in [3]. Extending 
the proof to deal with awareness is also straightforward once we add to the 
canonical model the canonical awareness sets A;(w) = {y(x) : Aip(x£) € w}, 
where w stands for a canonical world. For example, consider, in the proof 
of the truth lemma, the case in which the inductive formula has the form 
Ajw: if Aip € w, then, by definition of the canonical A;(w), y € A;(w) 
or (M,w) =o Aip, and vice versa. Note that axioms Al—A7 ensure that 
awareness is weakly generated by atomic formulas. 


3.4 Decidability 


This section is based on Wolter and Zakharyashev’s decidability proof for 
the monodic fragment of first-order multi-modal logics interpreted over 
Kripke structures. The proof is here generalized to neighborhood models 
with awareness. The monodic fragment of first-order modal logic is based on 
the restricted language in which formulas in the scope of a modal operator 
have at most one free variable. The idea of the proof is the following: 

We can decide whether the monodic formula ọ is valid, provided that 
we can decide whether a certain classical first-order formula a is valid. This 
is because, by answering the satisfiability problem for a, we can construct 
a so-called “quasi-model” for y. A “quasi-model” satisfying y, as it will be 
clear in the following, exists if and only if a neighborhood model satisfying 
y exists. Furthermore, if a model satisfying y exists, then it is possible to 
effectively build a “quasi-model” for y. Hence the validity problem in the 
monodic fragment of first-order modal logic can be reduced to the validity 
problem in classical first-order logic. It follows that the intersection of the 
monodic fragment and (several) decidable fragments of first-order classical 
logic is decidable”*. 

In carrying the proof over to neighborhood structures with awareness, a 
few adjustments of the original argument are necessary. First, the overall 
proof makes use of special functions called runs. Such functions serve the 


24 The mosaic technique on which the proof of the existence of an effective criterion for 
the validity problem is based was introduced by Németi (cf. for example [34]). The 
proof on which this subsection is based can be found in [40]. The same technique is 
used in [39] to show that first-order common knowledge logics are complete. For a 
more compact textbook exposition of the proof, cf. [4]. 
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purpose of encoding the modal content of the structure. Since the modal 
operators are now interpreted through neighborhoods rather than through 
accessibility relations, the definitions of runs and of related notions have to 
be modified accordingly. Second, the proof of Theorem 3.1 accounts for the 
cases of the modal operators introduced in the present setup (i.e., awareness 
and explicit knowledge operators). Third, a suitable notion of “unwinding” 
a neighborhoods structure has to be found in order to prove Lemma 3.2. 
Fourth, the use of neighborhood structures slightly modifies the argument 
of the left to right direction in Theorem 3.3. In the rest of this subsection, 
I shall offer the main argument and definitions, relegating the more formal 
proofs to the Appendix. 

Fix a monodic formula y. For any subformula 0;¢(«) of p, let Po,y(«) 
be a predicate symbol not occurring in 7, where 0; = { Kj, Ai, Xi}. Po,y(2) 
has arity 1 if y(x) has a free variable, 0 otherwise, and it is called the 
surrogate of y(x). For any subformula ~ of p, define Y to be the formula 
obtained by replacing the modal subformulas of w not in the scope of another 
modal operator with their surrogates, and call Y% the reduct of w. 

Define sub, y = {v[x/y] : w(y) € suby}, where suby is the closure 
under negation of the set of subformulas of y. Define a type t for y as any 
boolean saturated subset of sub, y?°, i.e., such that, (i) for all Y € sub, ¢, 
ay E t iff y Zt; and (ii) for all PA x € subs y, Y A x E t iff y and x belong 
to t7°. Types t and t' are said to agree on subo ọ (the set of subsentences 
of vy) if tN subo y = t’ N subo y. 

The goal is to encode a neighborhood model satisfying y into a quasi- 
model for y. The first step consists in coding the worlds of the neighborhood 
model. Define a world candidate to be the set T of y-types that agree on 
subo y. Consider now a first-order structure D = (D, PP,...), let a € D and 
define t? (a) = {y € sub, y: D H Yla]}, where H stands for the classical 
satisfiability relation. It easily follows from the semantics of — and A that t? 
is a type for y. A realizable world candidate is the set T = {tP (a) : a € D}. 
Notice that T is a realizable world candidate iff a formula ar is satisfiable 
in a first-order structure, where ar is 


VAN Jæt(x) A Yr VV t(x), (ar) 


tET tET 
in which t(x) := Aides Y(a). 


25 Or, equivalently, as “any subset of sub, y such that {7 : Y € t} is maximal consistent”, 
where 4% is any subformula of y: cf. [4]. 

26 For example, consider y := K;P(y) A X:3zR(y, z). Then subz y is the set {K;P(x) A 
Xi3zR(x, z), Ki P(x), P(x), X;3zR(x, z), dzR(a, z)}, along with the negation of such 
formulas. Some of the types for y are ®U {AzR(a, z), P(x)}, ®U {~73z R(x, z), P(x)}, 
etc.; = U {AzR(a, z), P(x)}, = U {AzR(a, z), =P(x)} etc., where ® = {Ki P(x) A 
Xi3zR(x, z), Ki P(x), XidzR(a, z)} and ~® = {~y : Y E ®}. 
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Intuitively, the formula says that all the reducts of the formulas in every 
type t € T are realized through some assignment in the first-order structure, 
while all assignments in the structure realize the reducts of the formulas in 
some type t € T. The existence of the satisfiability criterion for y will 
ultimately be given modulo the decidability of ar for each realizable world 
candidate in the model, hence the restriction to the monodic fragment based 
on a decidable fragment of predicate logic. 

Set a neighborhood frame with awareness F = (W,Ni,...,Nn,A1,---; 
An). We can associate each world w in W to a corresponding realizable 
world-candidate by taking the set of types for y that are realized in w. 
Let f be a map from each w € W to the corresponding realizable world 
candidates Tu. Define a run as a function form W to the set of all types of 
y such that 


(i) r(w) € Tv, 


) 

(ii) if Kiy € sub, y, then, Kiy € r(w) iff {v : Yy € r(v)} € N(w), 

(iii) if Aja € sub, y, then Aip € r(w) iff Y € Ai(w), 

(iv) if Xip € sub, y, then Xip € r(w) iff {v : y € r(v)} € N(w) and 
p E€ Ailw). 


Runs are the functions that encode the “modal content” of the neighbor- 
hood structure satisfying ọ that was lost in the reducts 7, so that it can be 
restored when constructing a neighborhood model based on the quasi-model 
for yp. 

Finally, define a quasi-model for y as the pair (F, f}, where f is a map 
from each w € W to the set of realizable world candidates for w, such that, 
for all w € W and t € T, there exists a run on F whose value for w is t. 
We say that a quasi-model satisfies ọ iff there exists a w such that y € t for 
some t € Tw. We can now prove the following 


Theorem 3.1. The monodic sentence y is satisfiable in a neighborhood 
structure M based on F iff y is satisfiable in a quasi-model for y based 
on F. 


Proof. See Appendix, Section A.1. Q.E.D. 


It is now possible to show that an effective satisfiability criterion for p 
exists by representing quasi-models through (possibly infinite) mosaics of 
repeating finite patterns called blocks?”. 


27 For ease of exposition and without loss of generality, from now on attention will be 
restricted to models with a single agent. 
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Recall that quasi-models are based on neighborhood frames. We restrict 
now our attention to monotonic frames? and say that a quasi-model for 
y is a tree quasi-model if it is based on a tree-like neighborhood frame. 
Section A.2 of the Appendix, drawing from [17], describes how a monotonic 
neighborhood model can be unravelled in a tree-like model. Hence, 


Lemma 3.2. A (monodic) formula ¢ is satisfiable iff it is satisfiable in a 
tree quasi-model for ọ at its root. 


Proof. The lemma stems obviously from the unravelling procedure described 
in the Appendix, Section A.2. Q.E.D. 


We now need to define the notion of a block for y. We shall then be 
able to represent quasi-models as structures repeating a finite set of blocks. 
Consider a finite tree-like structure (called a bouquet) (Fn, f), based on 
Wn = {w0,..., Wn}, rooted in wo, such that no world in the structure but 
wo has a nonempty neighborhood. 

A root-saturated weak run is a function r from Wn to the set of types 
for y such that 


(i) r(wn) € Ton» 

(ii) if Kiy € sub, y, then, Kiy € r(wo) iff {v : y € r(v)} € N(w), 

(iii) if Aja € sub, y, then Aip € r(wo) iff y € Ai(w), 
) 


(iv) if Xy € sub, y, then Xiy € r(wo) iff {v : Y € r(v)} € N(w) and 


A block is a bouquet (Fn, fn), where fn is a map from each w € Wn to 
the set of realizable world candidates for w such that, for each w € Wn and 
t € T, there exists a root-saturated weak run whose value for w is t. We 
say that y is satisfied in a block (Fn, fn) iff there exists a w such that y € t 
for some t € Ty. 

Finally, a satisfying set for y is a set S of blocks such that (i) it contains 
a block with root wo such that y € t for all t € Tw, (that is, wo satisfies 
y), and (ii) for every realizable world candidate in every block of S, there 
exists a block in S rooted in such a realizable world candidate. 


It is now possible to prove the following 


Theorem 3.3. A monodic sentence y is satisfiable iff there exists a satis- 
fying set for y, whose blocks contain a finite number of elements. 
28 This restriction yields a less general proof, since it implies that the decidability result 


does not hold for non-monotonic systems. Given the intended interpretation of the 
modalities (high-probability operators, cf. Section 1), the restriction is not problematic. 
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Proof. See Appendix, Section A.3. Q.E.D. 


The effective satisfiability criterion now follows: 


Corollary 3.4. Let Lm be the monodic fragment and L’, C Lm. Suppose 
that for y € Li, there is an algorithm deciding whether a world-candidate 
for ọ is realizable (that is, whether the classical first-order formula ar is 
satisfiable.) Then the fragment £/,9 QEM is decidable. 


In particular, the monodic fragment is decidable if it is based on the 
two- (one-) variable fragment, on the monadic fragment, and on the guarded 
fragment of classical predicate logic (cf. [40]). 
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Appendix 
A.1 Proof of Theorem 3.1 
Proof. |=] Let M be a neighborhood structure satisfying y. Construct a 


quasi-model as follows: Define the map f by stipulating that 
t? = {y € subs y : (M,w) Ho Y}, where a € D and o(x) = a, 
Tw = {t} :a € D}, 


and let, for alla € D and w € W, r(w) = t”. We need to show that 
r is a run in (F, f}. For (i), r(w) = t” € Ty by construction. For (ii), 
Kiab(x) € r(w) iff (M,w) Ho Kiyle) iff {v : (M, v) Ho V(a2)} € Mi(w) 
but, for all a € D, t? = {w € subs ọ : (M,v) Ho Y(x)} and r(v) = t} by 
definition, thus {v : (M, v) Eo y(x)} = {v : y(x) € r(v)}, as desired. For 
(iii), (M, w) Ho Ait) iff Y € Ai(w), hence Aj € r(w) if y € Ai(w). The 
case for (iv) follows immediately from (ii) and (iii). 


Na 


[<=] Fix a cardinal «x > No that exceeds the cardinality of the set Q of all 
runs in the quasi-model. Set D = {(r, £): r € 0,€ < K}. Recall that a world 
candidate T is realizable iff the first-order formula ær is satisfiable in a first- 
order structure and notice that, since the language we are using does not 
comprehend equality, it follows from standard classical model theory that we 
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can consider the first-order structure D to be of arbitrary infinite cardinality 
k >No. Hence, for every w E€ W, there exists a first-order structure I(w) 
with domain D that realizes the world candidate f(w). Notice that the 
elements in the domain of such structures are specific runs indexed by the 
cardinal £. Let? r(w) = {4 € subs y: I(w) = ¥[(r, €)]} for all r € Q and 
E<K. 

Let the neighborhood structure be M = (W,N,,...,Nn,A1,---;An, 
D,TI) and let ø be an arbitrary assignment in D. For all y € suby and 
w € W, we show by induction that 


I(w) Ho Y iff (M, w) Ho Y. 


The basis is straightforward, since 7 = 7 when y is an atom. The 
inductive step for the nonmodal connectives follows from the observation 
that DAW =Y AY, ad = 70, Yzy = Vey, and the induction hypothesis. 
Consider now the modal cases. Fix o(y) = (r,€) First, let y := Kix(y). 
The reduct of w is the first-order formula Px,,(y). We have that 


I(w) Fo Prix(y) iff (construction of quasi-model) 
Kix (y) € r(w) iff (definition of run) 

{v : x(y) € r(v)} € Mi(w) iff (definition of r(v)) 

{v : I(v) = X(y)} € Mi(w) iff (induction hypothesis) 

{u: (M,v) Ho x(y)} E Ni(w) iff (semantics) 

(M, w) Fo Kix(y). 


Second, let y := Ajx(y). The reduct of w is the first-order formula 
Pa,x(y). Then, 


NK 


I(w) Fo Pa,x(y) iff (construction of quasi-model) 
Aix(y) € r(w) iff (definition of run and of r(w)) 


x(y) € Ai(w) iff (semantics) 
(M, w) FO Aix. 
Finally, let Y := X;ix(y). We have that I(w) =o Pyx:x(y) iff I(w) Ho 
Pxix(y) \ Pa:x(y), which follows from the two cases just shown. Q.E.D. 


A.2 Unravelling a neighborhood structure 

In this subsection I describe the procedure defined in [17] to unravel a 
core-complete, monotonic model M into a monotonic model whose core 
neighborhoods give rise to a tree-like structure that is bisimilar to M. 
Definition A.1. (Core-complete models) Let the core N° of N be defined 
by X € N°(w) iff X € N(w) and for all Xo C X, Xo Z N@(w). Let M be a 
neighborhood model. M is core-complete if, for all w € W and X C W, If 
X € N(w), then there exists a C € N°(w) such that C C X. 


29 For a proof that this assumption is legitimate, cf. [39]. 
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The idea is that we can unravel a core-complete, monotonic neighbor- 
hood structure (with awareness) into a core-complete neighborhood which 
is rooted and whose core, in a sense that will be made precise below, con- 
tains no cycles and has unique, disjoint neighborhoods. The unravelling 
procedure described above is given in [17]. 


Define the following objects: 


Definition A.2. Let M be a core-complete monotonic model. For any 
X C W, define NS(X) and Su (X) as the union, for all n > 0, of the objects 
defined by double recursion as: 


So(X) = X, NEX) = U N“(2) 
LEX 
Soe U e N&(X)= U N 
YENE (X) LESn(X) 


In words, we start with a neighborhood X, and take MẸ(X) to be the 
core neighborhoods of the worlds in X. We add all worlds in such core 
neighborhoods to the space set of the following stage in the inductive con- 
struction, and then consider all core neighborhoods of all such worlds, etc. 
If the set of all worlds in a model M is yielded by S,,({w}), then M is said 
to be a rooted model. 

We can now define a tree-like neighborhood model as follows: 


Definition A.3. Let M be a core-complete monotonic neighborhood model 
with awareness, and let wo E€ W. Then M,,, is a tree-like model if: 


(i) W = S..({wo}); 
(ii) For all w € W, w € Unso Sn({w}); 


(iii) For all w,w’,v € W and all Xo, X1 C W: If v € Xo E€ N“(w) and 
VE Xı € N“(w"), then Xo = Xı and Wo = w1. 


That is to say, (i) M is rooted in wo; (ii) w does not occur in any 
core neighborhood “below” w, thus there are no cycles; and (iii) all core 
neighborhoods are unique and disjoint. 

The neighborhood model M = (W,N, A, T) can now be unravelled into 
the model Mwa = (Wwa, Nwo, Awos Two) as follows: 


(1) Define its universe Ww as 


Wowo = {(w0oXiw ...Xnwn):n>0 
and for each | = 1, e’ Me Xı E N (w1), w € Xi} 
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In English, Ww, contains all sequences of worlds and neighborhoods 
obtained by beginning with wo and appending to each state w; the sets be- 
longing to its neighborhood, and by further appending the worlds contained 
in the element of the neighborhood under consideration. For example, if the 
model contains a world w whose neighborhood contains the set {x,y} the 
space of the unravelled model rooted on w contains also worlds w{x, y}x 
and w{x,y}y. 

In order to define the neighborhoods of the unravelled model, we need 
to define two maps pre and last as: 


pre : (woX1wW1...XnWn) > (woX1u1... Xn) 
last : (woX1W1...XnWn) > wn. 


(2) Define now a neighborhood function Ng, : Wwo > P(P(Ww.)) as 
follows, with s’ € W,,, and Y C Wu: 


Y € NE, C5) iff for all Y € Y and some X € P(W), 


pre(y) = $X and U last(y ) = X € N(last(5)) 
Gey 
Thus, every neighborhood in the original model N (last(s’)) originates 
exactly one neighborhood Y in NE (5) and all sets Y are disjoint. Closing 
the core neighborhoods under supersets yields now the neighborhoods of 
the monotonic model. 


(3) Define an awareness function Aw, such that y € Aw (W) iff y € 
A(last(@w )). 


(4) Finally, we take 7,,(s) to agree with m(last(S’)). 


It follows that (M,(wo)) = iff (Mus, Wo) H y, since we can root 
the unravelled model on the world w satisfying y in the original model. 
Moreover, if, as we are assuming, the original model M is core-complete, 
the core neighborhoods of the unravelled tree-like model still give rise to a 
model that is bisimilar to M. 


A.3 Proof of Theorem 3.3 


Proof. |=] If y is satisfiable, by the lemma above there exists a tree quasi- 
model satisfying y at its root. For all X € N(w), with w € W, either there 
are sufficiently many sets Y, called twins of X such that Y € P(W), the 
submodel generated by Y is isomorphic to the one generated by X and, for 
alla € X and all y € Y, f(x) = f(y), or we can make sure that such is the 
case by duplicating, as many time as needed, the worlds in the neighborhood 
X in a way that the resulting structure is equivalent to the given one, and 
thus still a quasi-model for ọ. 
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For every w € W, now, construct a finite block By, = (Fw, fw) with 
Fy = (Ww, Nw) as follows: 

For every t € Ty, fix a run in the quasi-model such that r(w) = t. For 
every Kw € sub, y such that Kw ¢ r(w), we select an X € P(W) such that 
X € N(w) and there exists v € X such that w(x) ¢ r(v) and put it in an 
auxiliary set Sel(w) along with one of its twins Y. Take W,, to be w along 
with all selected worlds, Nw to be the restriction of M to Ww, and fy to 
be the restriction of f to Wy. The resulting structure B,, is a block for y 
since it is based on a bouquet (of depth 1) and it is a subquasi-model of the 
original quasi-model for vy”. 

We now illustrate precisely the construction sketched above, and show 
that for all w € W and tw € Tẹ there exists a root-saturated weak run 
coming through tw. For this purpose, let u € Ww,t € Tu and r be a weak 
run such that r(u) = t. Consider the type r(w) and the set C = {x := 
Kw €sub,¢: x Z r(w)}. For any such x, there exists a weak run ry such 
that: (i) ry(w) = r(w), (ii) yY g r(wy) for some world wy E€ Ww in some 
selected X € N(w) and (iii) u Æ w, wy. Define now, for any w’ € Wy, the 
root-saturated weak run r’ such that (a) r(w’) if w # w, wy for all x € C, 
and (b) r,(w’) otherwise. 

The satisfying set for y is now obtained by taking the blocks Bu for each 
w € W, each block containing at most 2-| sub, y| -2!**>s #! neighborhoods. 


[<=] If S is a satisfying set for y, we can inductively construct a quasi- 
model for y as the limit of a sequence of (weak) quasi-models (Fn, fn) with 
n=1,2,...and Fa = (Wn, Nn, An). The basis of the inductive definition is 
the quasi-model mı, which is a block in S satisfying ọ at its root. Assuming 
we have defined the quasi-model mz, let mz4 1 be defined as follows: For 
each w € Wm — Wm-1ı (where Wo is the root of Fı) select a block 6’, 
such that f,(w) = fw/w’ and append the selected blocks to the appropriate 
worlds in mz. We can then take the desired quasi-model to be the limit of 


30 To see this, consider, for instance, the case that Ky € t = r(w). Then, for all 
v E€ Nw(w), Y € r(v). Now, if there does not exist any type t’ such that Kw ¢ t’, we 
are done. If there is such a type, however, there exists a run r’ such that Ky ¢ r'(w) 
and we select sets X, Y € P(W) such that they belong to N (w), {v : (M,v) = y} = X 
and, for all x,y in X,Y respectively, y belongs to both r(x) and r(y). The idea of 
the construction, now, is to define a further run, which goes through the types of, 
say, x that does not contain 7, making sure that it is ‘root-saturated.’ Notice that 
blocks constructed this way are always quasi-models, since they are root-saturated 
weak quasi-models of depth one. However, if we consider also, as it is done in [40], 
the transitive and reflexive closure of the neighborhood functions (a sort of “common 
knowledge” operator), then resulting bouquets have depth larger than 1, and blocks 
are indeed based on weak quasi-models. 
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the sequence thus constructed by defining the elements in (W,N, A, f) as 


Wea Ma AS Ae MSA ES 


n>1 n>1 n>1 n>1 


Clearly, the resulting structure is based on an awareness neighborhood 
frame, and f is a map from worlds in W to their corresponding sets of world 
candidates. It remains to show that, for each world and type, there exists 
a run in the quasi-model coming through that type. We define such runs 
inductively, taking r! to be an arbitrary (weak) run in mı. Suppose r* has 
already been defined: Consider, for each w € Wk — Wk—1, runs ry(w) and 
such that r*(w) = ry (w). Now, for each w € Wk+1 — Wp take r*+! to be (i) 
r*(w’) iff w € We and (ii) ry (w’) iff w € Wy — We. Define r as U,. 7°. 
The constructed function r is a run in the limit quasi-model since, at each 
stage k of the construction, it has been “added” to r a root-saturated run 
rF hence, in the limit, r is saturated at each w € W. Q.E.D. 
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